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PREFACE 


This  series  of  lectures  on  optimization  techniques 
was  originally  prepared  for  presentation  to  the  partici¬ 
pants  of  the  1959  Associate  Staff  Training  Program.  Its 
objectives  were  two-fold:  (a)  to  introduce  engineers  to 
the  usefulness  of  mathematical  techniques  and  mathema¬ 
ticians  to  the  applicability  of  their  techniques  to  con¬ 
crete  engineering  problems  and  (b)  to  introduce  all  the 
trainees  to  the  very  important  concept  of  optimization — 
a  concept  that  currently  pervades  virtually  every  area  of 
advanced  missile  engineering. 

Because  these  lectures  do  form  a  sound,  basic  intro¬ 
duction  to  the  important  field  of  optimization  techniques, 
they  are  being  issued  as  a  CF  report  to  provide  wider  dis¬ 
tribution  of  this  material  as  a  reference  work  for  Labora¬ 
tory  personnel. 


V .  M .  Root 

Training  Program  Supervisor 
August  1960 
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I  .  INTRODUCTION 


A  somewhat  negative  approach  to  the  principle  of  opti¬ 
mization  was  first  scribbled  centuries  ago, among  other 
phrases,  on  the  walls  of  an  ancient  Roman  bath  house.  Re¬ 
ferring  to  a  choice  between  two  Roman  aspirants  for  emperor, 
:t  read  -  "De  duobus  malis,  minus  est  semper  aligendum" 

Of  two  evils,  always  choose  the  lesser!. 

Since  that  time  optimization  has  been  carried  on  under 
many  guises.  In  physics,  many  different  minimal  principles 
have  been  enunciated,  describing  natural  phenomena  in  the 
fields  of  optics  and  classical  mechanics.  The  field  of  sta¬ 
tistics  contains  various  principles  termed  "maximum  likeli¬ 
hood,"  "minimum  loss,"  and  "least  squares,"  while  economics 
contributes  "maximum  profit,  minimum  cost,  maximum  use  of 
resources,  minimum  effort,"  in  a  coherent  effort  to  increase 
the  long  run  capital  gain  in  some  manner. 

Enlarging  our  viev.^oint  to  include  the  most  general 
aspects,  we  note  that  many  operational  problems  are  of  this 
sort . 

1.  They  have  a  variety  of  acceptable  solutions  (by 
some  specific  criteria  of  acceptability) . 

2.  Among  these  solutions  one  wishes  to  select  the 
best  or  optimal  solution  (by  some  specific 
criteria  of  being  best  or  optimal) . 


Thus,  one  formulates  the  problem  mathematically  with 
the  twin  objectives  of  providing  an  accurate  description  and 
also  manipulating  the  mathematical  model  to  obtain  an  ex¬ 
tremum  . 

An  extremum,  or  extreme  value,  is  a  value  of  a  function 
which  is  either  a  maximum  or  a  minimum.  Optimum  is  that 
particular  type  of  extremum  desired  for  the  "problem. 
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II.  FULL,  INCOMPLETE ,  AND  SUBOPTIMIZATION 


Full  optimization  of  a  given  problem  can  be  quite  an 
extensive  undertaking.  It  requires 


1.  simultaneous  consideration  of  all  possible 
alternatives  at  all  levels; 

2.  consideration  of  the  probable  impacts  of  all 
events  not  under  the  optimizer's  control;  and 

3.  maximization,  subject  to  possible  constraints, 

of  some  utility  function  or  measure  of  effective¬ 
ness  . 


Failure  to  achieve  full  optimization  results  in  "incom¬ 
plete  optimization,"  of  which  a  special  variety  is  termed 
"suboptimization. " 

Suboptimization  is  a  case  of  optimization  for  one 
phase  of  an  operation  or  a  problem,  without  including  every 
factor  which  has  an  effect,  either  obvious  or  indirect. 
Frequently,  it  consists  of  merely  reducing  the  number  of  ob¬ 
jectives  . 

The  suboptimization  approach  is  useful  when  neither  the 
problem  formulation  nor  the  available  techniques  permit  one 
to  obtain  a  reasonable  answer.  In  most  practical  cases, 
suboptimization  is  the  only  resort  in  solving  the  problem. 
Although  a  full  optimization  is  not  obtained,  it  at  least  pro¬ 
vides  a  rational  technique  for  approaching  the  optimum. 

Suboptimization  is  often  necessary  because  of  economic 
and  practical  considerations,  the  finiteness  of  time,  and  the 
difficulty  of  obtaining  sensible  answers  in  a  hurry. 

However,  there  is  a  major  fallacy  to  be  guarded  against. 
Suboptimization  of  all  elements  does  not  necessarily  ensure 
attainment  of  full  optimization,  i.e.~  an  over-all  optimum. 

For  example,  in  a  large  business,  the  sales  force  endeavors 
to  increase  sales  of  all  items,  although  the  profit  on  each 
item  may  differ. 

The  production  group  resists  changes  to  new  products. 

The  comptroller  wishes  to  reduce  inventory  level  so  as  to 
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free  capital  and  decrease  storage  costs,  etc.  It  is  not 
difficult  to  suboptimize  all  these  divisions  separately,  so 
that  each  is  running  smoothly  and  effectively,  but  a  more 
painstaking  effort  is  necessary  to  balance  the  tendencies  of 
different  parts  of  a  large  organization  and  ensure  they  all 
mesh  together. 

System  design*  is  sometimes  described  as  the  process 
of  attaining  a  full  optimization.  The  need  for  system  design 
arises  from  this  very  fact  that  suboptimization  of  all  com¬ 
ponents  does  not  necessarily  improve  system  performance. 

An  excellent  example  of  this  has  actually  appeared  in 
the  following  concrete  situation  involving  target  tracking 
by  a  homing  missile. 

The  use  of  doppler  information  to  aid  range  gate  track¬ 
ing  has  long  been  known.  In  obtaining  the  optimum  over-all 
system,  usually  the  range  and  speed  tracking  loops  are 
separately  optimized  according  to  some  criterion  and  then  the 
speed  gate  is  connected  to  the  range  gate  to  further  reduce 
the  range  error. 

The  resulting  system  may  approach  the  optimum  over-all 
system;  however,  the  best  use  of  the  additional  information 
consisting  of  the  correlation  between  speed  and  range  has  not 
been  made. 

The  optimum  circuit  consists  of  two  optimum  loops  for 
range  and  doppler  information  tied  together  in  an  optimum 
fashion.  Consequently,  we  require  optimum  gaTns  connecting 
one  circuit  to  the  other  as  well  as  gains  in  the  individual 
circuit  loops. 


III.  OPTIMIZATION  TECHNIQUES  AND  CONSTRAINTS 


We  are  all  familiar  with  the  optimization  problem  of 
finding,  among  all  the  paths  between  two  points  in  the  plane, 
the  shortest  path.  It  can  be  shown  that  it  is  a  straight  line. 

Consider  the  extension,  where  the  two  points  lie  on 
the  same  side  of  a  given  straight  line,  and  the  problem  is  to 


A  system  is  an  integrated  assembly  of  interacting  elements, 
designed  to  carry  out  cooperatively  a  predetermined  function."  - 
Dr.  R.  E.  Gibson. 
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determine  the  shortest  path  between  the  two  points  that 
touches  the  given  line.  This  added  condition  is  a  constraint 
on  the  problem. 


(The  solution  is  not  difficult  in  this  case.  Reflect 
one  of  the  two  points  across  the  line,  and  connect  the  other 
original  point  and  the  reflected  point  with  a  straight  line, 
as  shown.  Then  ACB  is  the  shortest  path  satisfying  the  con¬ 
straint.) 


B 


Optimization  of  some  function  of  a  number  of  variables 
subject  to  boundary  conditions  whic*>  limit  the  variables'  range 
is  of  greater  importance  and  frequency  of  occurrence  than  opti¬ 
mization  without  constraints.  We  usually  are  not  interested 
in  relative  maxima  or  minima,  but  rather  maxima  or  minima  over 
a  prescribed  range.  This  entails  additional  restrictions. 

For  example,  in  the  range  a<  x  <  b, 
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there  is  a  maximum  at  x  -  b,  but  not  at  x^  or  x.j  which  lie 

outside  the  range  of  interest.  (This  so  called  end  point 
maximum  will  be  discussed  more  fully  later.)  There  is  a  mini¬ 
mum  at  x g  within  the  range. 

The  difficulties  in  the  more  interesting  problem  of  opti 
mization  subject  to  constraints  occur  in  many  cases  where  the 
maximum  or  minimum  cannot  be  obtained  by  ordinary  differentia¬ 
tion  because  either  it  does  not  lie  within  the  region  defined 
by  the  constraint  set  or  else  the  derivative  is  discontinuous 
inside  the  region. 

If  the  function  to  be  optimized  is  continuous,  then  the 
extremum  lies  either  in  the  region  or  on  the  boundary.  If  it 
lies  within  the  range,  regular  methods  of  finding  maxima  and 
minima  apply  as  if  the  inequalities  were  not  present. 

Mathematical  constraints  are  of  two  types:  equations  and 
inequalities.  In  a  typical  problem,  the  constraints  may  be 
all  one  or  the  other,  or  a  mixture.  Considerable  difference 
exists  in  the  techniques  applied  to  problems  in  which  all  con¬ 
straints  are  equations  or  else  all  inequalities. 

If  all  constraints  are  equations,  then  in  principle  each 
constraint  equation  can  be  solved  for  one  of  the  variables  and 
this  substituted  to  reduce  the  dimension  of  the  problem  by  one 
For  example,  to  find  the  extrema  of  f(x,  y)  subject  to  the  con 
straint  ^(x,y)  -  0,  we  solve  for  y  from  <£(x,  y)  -  0  to  obtain 
y  -  iKx)  .  Then  substitute  into  Z  -  f(x,y)  and  extremalize 


Z  -  f  [x,  tKx)] 


as  a  one-dimensional  problem. 

An  inequality  constraint  does  not  make  it  possible  to 
"eliminate"  any  variables;  it  merely  restricts  the  range  of 
variability  in  one  dimension.  Thus,  if  the  extreme  should  be 
on  the  boundary  or  end  point  of  the  range,  a  different  method 
for  extremalization  is  necessary. 

An  end-point  technique  is  as  follows:  Given  a  function 
f(x)  defined  for  a  51  x  <b,  and  let  f(x)  have  a  derivative. 
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Then  f (x)  has  a  maximum  at  x  -  a  if  f'(a)<  0,  and  a 

maximum  at  x  -  b  if  f '  (b)  >  0, 

It  will  have  a  minimum  at  x  -  a  if  f'(a)  >  0,  and  a 
minimum  at  x  *=  b  if  f '  (b)  <  0. 

It  will  occasionally  happen  that  f’(x)  becomes  dis¬ 
continuous  for  some  isolated  value  of  x,  ant’,  if  the  discon¬ 
tinuity  is  accompanied  by  a  change  of  sign  as  x  increases 
thru  the  value  in  question,  we  shall  have  a  maximum  or  mini¬ 
mum,  as  shown. 


These  remarks  pertaining  to  one  variable  are  generaliz- 
able  to  two  or  more  variables,  where  the  boundaries  become 
curves  and  surfaces  and  their  intersections.  The  difficulties 
of  testing  increase  also. 

Actual  constraints,  which  will  be  discussed  elsewhere 
under  several  examples,  are  many  and  varied.  They  can  include: 
negative  production  not  allowable,  maximum  limits  on  storage 
capacity,  and  production  capacity  in  economic  problems.  Non- 
linearities  in  physical  systems,  such  as  saturation  and 
limiting,  can  be  described  mathematically  in  the  form  of  ine¬ 
qualities.  Noise  or  uncertainty  is  often  describable  in  terms 
of  a  probability  distribution  or  power  spectra.  Competitive 
or  game  theory  aspects,  such  as  opponents’  strategies,  must  be 
considered  in  many  cases. 

Physical  constraints,  involving  weight  and  size  are  by 
no  means  minor.  For  example,  an  optimum  cargo  ship  hull  de¬ 
sign  problem  separates  into  two  classes; 

1.  Designs  in  which  size  (displacement)  is  fixed,  as 
determined  by  available  power. 

2.  Designs  in  which  dimensions  (particularly  length) 
are  dixed,  as  determined  by  practical  considerations 
of  port  facilities,  such  as  dock  size. 
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Different  optimum  solutions  are  to  be  expected  in  each 

case . 


IV.  THE  CRITERIA  OF  OPTIMALITY 


A  major  difficulty  in  any  optimizing  problem  is  that  of 
selecting  a  criterion.  What  is  the  criterion  in  terms  of 
which  the  outcome  is  to  be  judged? 

In  game  theory,  the  sensible  object  of  a  player  is  to 
gain  as  much  from  the  game  as  he  can,  safely,  in  the  face  of 
a  skillful  opponent  who  is  pursuing  an  antithetical  goal. 

The  classical  criterion  for  optimizing  the  design  of  a  mechani¬ 
cal  device  has  been  to  maximize  the  output  energy  for  a  given 
input  energy.  In  the  case  of  vibration  of  a  continuous  system, 
the  optimum  damping  value  gives  the  least  resonant  amplitude. 

In  choosing  an  optimization  criterion,  several  aspects 
must  be  considered. 

First,  viewpoint  is  important.  For  example,  consider  a 
nonproduction  line  type  of  manufacture,  i.e.,  the  product  is 
generated  in  discrete  batches.  From  the  viewpoint  of  reducing 
storage  costs,  we  might  ask  "How  can  inventory  level  be  re¬ 
duced?"  But  a  wider  viewpoint  is  represented  by  the  question 
"What  is  the  optimum  size  of  inventory  with  respect  to  making 
a  profit?"  From  the  savings  resulting  from  larger  batch  size 
with  less  frequent  production,  it  might  be  that  the  inventory 
size  should  be  increased. 

Constraints  are  also  important  in  criteria  selection. 

For  an  airborne  digital  computer,  weight  and  size  are  fixed,  so 
that  any  criterion  for  optimal  choice  among  several  computers 
must  include  the  question  "Does  it  fit?" 

There  may  well  be  a  multiplicity  of  criteria  to  plague 
the  optimizer.  They  must  be  reduced  in  some  way  to  manageable 
proportions  by  grouping  them,  inter-relating  them,  or  just 
discarding  them.  Conversely,  one  of  the  main  reasons  that 
criteria  selection  is  difficult  is  the  fact  that  one  usually 
deals  with  either  incomplete  or  suboptimization.  In  most 
cases,  however,  the  performance  criterion  is  the  first  to  be 
examined . 

Consider  the  application  of  optimalizing  principles  to 
the  field  of  automatic  controls  for  an  aircraft  jet  propulsion 
system . 
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One  such  optimalizing  control  meters  the  input  fuel  flow 
to  an  engine  in  such  a  manner  as  to  produce  the  maximum  com¬ 
pressor  output  pressure  over  a  range  of  flight  conditions. 


Pressure 

Ratio 


1 


Fuel  Flow,  lbs/hr 


This  indicates  the  goal  is  to  maintain  delivery  of  thrust 
without  consideration  of  fuel  consumption. 

Clearly,  if  the  economical  use  of  fuel  is  the  primary 
purpose,  then  the  output  might  have  been  the  ratio  of  pressure 


Here  the  optimum  point  (a  maximum)  has  shifted  to  a 
lesser  fuel  flow.  Which  optimum  is  better?  No  statement  is 
possible.  Each  is  an  optimum  for  its  chosen  criterion. 

To  sum  up  the  essence  of  this  section,  the  optimal  ox 
consists  of 

all  meat  to  the  gourmand, 
all  hide  to  the  shoe  seller, 
and  all  hoof  to  the  glue  maker. 
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Therefore,  always  define  in  advance  your  chosen  criterion  of 
optimality  and  the  constraints. 


V.  VARIOUS  EXAMPLES 


Optimum  Coding  of  Information  -  Communication  and 

information  theory  abound  with  the  word  "optimum."  Having 
determined  a  cost  in  energy,  time,  or  money  of  transmitting 
each  one  of  a  set  of  symbols  (e.g.,  Morse  code),  what  is  the 
optimal  code  using  these  symbols  which  will  transmit  a  given 
amount  of  information  at  the  least  cost,  or  will  transmit 
information  at  a  given  rate  for  the  least  cost  per  unit  time? 
(If  noise  is  present,  the  cost  of  each  symbol  is  effectively 
increased . ) 

Thus,  in  an  optimum  coding  process,  we  try  to  produce  a 
set  of  symbols  (each  which  may  take  different  times  to  trans¬ 
mit)  to  be  sent  over  the  communication  channel  so  that  they 
will  all  occur  independently  and  with  equal  frequency.  This 
will  permit  a  message  to  be  encoded  in  such  a  way  as  to  utilize 
the  fixed  channel  capacity  in  an  efficient  manner. 

Sometimes  optimum  code  means  a  minimum-redundancy  code. 
This  is  a  code  which,  for  a  message  ensemble  consisting  of  a 
finite  number  of  members  and  for  a  given  number  of  coding 
symbols,  yields  the  lowest  possible  average  message  length. 

In  general,  the  optimum  code  alone  may  not  be  identical 
with  the  optimum  code  when  channel  characteristics  enter  as 
constra ints . 

Optimum  Programming  of  Computers  -  -The  speed  of  a 
storage  device ~  such  as  a  computer  memory,  is  measured  by  its 
access  time,  the  time  required  for  either  reading  or  writing 
access  to  the  first  location  required.  Access  can  be  random 
(each  bit  available  within  the  fixed  access  time)  in  the  case 
of  magnetic  cores,  or  it  can  be  cyclic  (in  which  the  access 
time  depends  on  where  the  bit  is  in  the  cycle)  as  in  the  case 
of  a  magnetic  drum  rotating  past  a  group  of  magnetic  reading/ 
writing  heads. 

If  a  computer  program  is  stored  wordwise  in  a  sequential 
manner  around  the  rotating  drum,  the  time  delays  involved  in 
reading  or  writing  in  one  memory  location  prevent  the  next  lo¬ 
cation  from  being  processed  immediately,  and  the  drum  must 
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spin  around  again  before  the  next  word  location  can  be  utilized, 
and  the  time  to  run  the  computer  program  is  considerably  ex¬ 
tended  by  this  waste  of  time. 


Optimal  programming  consists  of  interweaving  the  locations, 
i.e.,  arranging  the  program  to  permit  space  between  consecutive 
storage  locations.  This  will  permit  consecutive  reading  despite 
the  control  time  lags.  In  this  way  the  time  of  running  is  re¬ 
duced.  Computers  with  cyclic  memories  (drum,  discs  or  delay 
lines)  require  such  optimal  (or  minimal  latency,  as  it  is  also 
known)  programming  so  as  to  minimize  the  computer  time  required 
for  a  given  program.  (A  disadvantage  is  the  widescatter ing 
of  orders  through  the  drum  memory.  This  may  cause  considerable 
difficulty  when  several  programs  have  to  be  fitted  together.) 

A  first  approach  consists  of  spacing  the  words  a  fixed 
distance  apart,  regardless  of  the  unequal  length  of  time  differ¬ 
ent  commands  require  for  execution.  A  distinct  improvement  re¬ 
sults  when  the  orders  are  spaced  a  variable  distance  apart,  so 
that  a  location  on  the  drum  tends  to  be  passing  the  magnetic 
reading  head  when  it  is  needed  for  access. 

Optimum  programming  usually  does  pay,  but  not  always.  In 
order  to  pay,  the  computer  time  saved  must  exceed  the  additional 
programming  time  required  for  optimizing  (subject  to  the  quali¬ 
fying  discussion  at  the  end  of  this  paragraph) .  It  will  always 
pay  if  (1)  the  program  is  to  be  used  repeatedly  in  processing 
large  volumes  of  data  and  if  (2)  the  programming  can  be  done 
before  the  numerical  data  for  the  problem  is  available.  It  may 
pay  if  the  problem  is  a  long,  nonrepeatable  one,  with  a  large 
amount  of  data . 

(Qualifying  discussion:  From  a  cost  accounting  viewpoint, 
it  is  natural  to  minimize  the  large  expense  of  using  digital 
computers  by  emphasizing  the  importance  of  "minimum  machine  time" 
concept  as  a  programming  philosophy.  Since  the  cost  of  one  hour 
machine  time  is  approximately  equal  to  one  week  of  programmer's 
salary,  it  might  seem  reasonable  for  a  programmer  to  spend  one 
week's  effort  "optimizing"  the  program  to  save  more  than  one 
hour  running  time.  However,  more  often  than  not,  several 
scientists  or  engineers  are  held  up  for  a  week  awaiting  an  answer 
before  they  can  take  action  on  it.  In  the  broad  view,  it  may  be 
less  expensive  to  get  answers  to  the  sponsor  before  he  forgets 
the  problem,  even  if  it  costs  more  in  machine  running  time.) 

Optimizing  the  computer  program  can  be  both  tedious  and 
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valuable.  Can  a  machine  be  programmed  to  optimize  its  own 
programs?  Yes  —  but  certain  compromises  are  necessary,  princi¬ 
pally  the  problem  program  and  the  optimizing  program  must  both 
fit  in  the  memory  at  the  same  time.  This  limits  the  size  of 
the  problem  that  can  be  optimized. 


Opt imum  Interval  Tables  -  Instead  of  storing  tables  of 

function  values  in  a  digital  computer,  or  printing  subtables  of 
differences  on  each  page,  polynomial  approximations  of  the 
tabled  function  can  be  made,  valid  throughout  the  tabular  region, 
with  a  given  allowable  error.  These  polynomial  representations 
can  then  be  evaluated  to  reconstruct  the  function  for  any  argu¬ 
ment  in  the  region,  and  thus  include  interpolation  as  well  as 
tabulation . 

Thus,  we  can  assign  in  advance  both  the  degree  n  of  the 
approximating  polynomial  and  the  maximum  allowable  error  t  . 

For  example,  in  the  sketch  shown  below,  the  curve  y  —  f(x)  would 
be  replaced  by  straight  lines  (n  -  1)  such  that  the  error  every¬ 
where  is  less  than  a  prescribed  e  (this  fact  determines  the 
length  of  each  subinterval) . 


/y  =  f(x) 

/ 


Now  for  an  interpolation  polynomial  of  degree  n  and  allowed 
error  e .  the  computer  merely  stores  the  coefficients  m.,  b.  of 
each  interval  and  evaluates  the  linear  polynomial  corresponding 
to  the  argument  x  lying  within  the  interval. 

This  process,  termed  optimum  interval  interpolation  mini¬ 
mizes  the  number  of  subintervals  needed  over  the  entire  range. 
Function  tables  so  constructed  are  called  optimum  interval 
tables. 
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VI.  CALCULUS  OF  VARIATIONS 


How  does  an  optimization  problem  in  the  calculus  of 
variations  differ  from  an  extremal  problem  of  ordinary  calcu¬ 
lus? 

In  the  latter  case,  we  are  given  the  function  y  -  y(x). 

A  simple  problem  in  ordinary  calculus  is  to  find  a  value  x 
which  yields  a  minimum  or  maximum  value  of  y  -  y(x)  . 

One  step  removed  from  finding  the  extremum  of  a  differ¬ 
ential  function  is  the  basic  problem  of  the  calculus  of  varia¬ 
tions;  to  find  a  function  y  -  y(x),  instead  of  a  value  of  the 
variable  x,  which  makes  a  certain  definite  integral 


b 

I  -  J  f(x,  y,  y')  dx 
a 


take  on  a  maximum  or  a  minimum.  Note  we  cannot  integrate 
directly,  because  y  is  not  known  as  a  function  of  x,  hence 
f(x,  y,  y')  is  not  known  as  a  function  of  x.  Thus,  the  ordi¬ 
nary  methods  of  solving  maxima  and  minima  problems  do  not 
apply. 


Geometrically,  the  calculus  of  variations  deals  with  the 
problem  of  finding  paths  of  integration  for  which  integrals  ad¬ 
mit  maximum  or  minimum  values.  Solutions  may  be  either  con¬ 
tinuous  or  discontinuous  in  the  first  derivative,  or  both,  i.e., 
"corners"  may  exist  in  the  path,  occurring  at  the  junction  of 
different  continuous  arcs. 

Extensions  to  the  basic  problem  include  the  presence  of 
higher  order  derivatives  in  the  integrand,  multiple  integrals 
involving  partial  derivatives  in  the  integrand,  variable 
limits  of  integration,  and  constraints  represented  by  the  re¬ 
quirement  that  another  integral,  involving  the  same  variables, 
has  a  constant  value.  Classical  applications  of  the  calculus 
of  variations  include  the  problems  of  finding  the  minimum  sur¬ 
face  of  revolution,  the  maximum  solid  of  revolution,  least 
action,  solids  of  minimum  resistance  with  and  without  the  con¬ 
straint  of  given  volume.  (A  recent  technique,  called  "Dynamic 
Programming"  is  a  powerful  computational  approach  to  both 
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classical  and  nonclassical  problems  in  the  calculus  of  varia¬ 
tions  . ) 


\  :  .  OPTIMIZATION  AND  ROCKETS 


The  field  of  rocketry  provides  an  exceedingly  ripe  area 
for  both  optimization  criteria  and  mathematical  techniques 
involving  the  calculus  of  variations. 

For  example,  consider  the  problem  of  specifying  the 
rocket  trajectory.  Different  methods  of  control  exist  in  flight. 
The  thrust  may  be  varied  in  magnitude,  and  often  the  thrust 
direction  is  variable.  These  changes  affect  the  flight  trajec¬ 
tory,  there  may  exist  some  single  goal,  such  as  attainment  of 
long  range  with  minimum  expenditure  of  propellent  and  struc¬ 
tural  weight,  or  attainment  of  some  altitude  in  a  reasonable 
time.  It  is  of  interest  to  find  out  how  to  adjust  the  avail¬ 
able  control  so  as  to  optimize  the  trajectory  in  the  sense  of 
maximizing  or  minimizing  some  function  such  as  range  or  time 
subject  to  constraints  such  as  fuel  consumed  or  altitude 
achieved.  Since  the  controls  can  usually  be  varied  at  will 
over  a  continuous  range  of  values,  such  trajectory  problems  be¬ 
long  to  the  calculus  of  variations. 

Over  the  past  years,  there  have  been  numerous  applica 
tions  of  the  calculus  of  variations  (as  well  as  other  techniques) 
to  optimization  of  rocket  problems.  Some  of  the  many  topics 
published  include  the  following: 

1.  Either  maximize  the  range,  altitude  or  some  other 
property  for  a  given  fuel  consumption,  or  specify 
such  a  property  and  seek  to  minimize  the  fuel 
consumed . 

2.  Program  the  exhaust  velocity  in  an  optimal  manner  so 
as  to  provide  the  most  efficient  utilization  of  the 
fuel. 

3.  Determine  the  optimum  thrust  direction  of  a  rocket 
fired  from  a  fighter  plane  pursuing  a  constant 
velocity  target  in  order  to  maximize  the  initial 
missile-target  range. 

4.  Determine  what  value  of  payload  that  will  give  maxi¬ 
mum  kinetic  energy  for  a  rocket  of  fixed  structural 
and  propellent  weights.  An  optimum  ratio  of  payload 
to  structure  exists  for  every  value  of  the  propellent 
ratio. 
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5.  Obtain  optimum  staging  techniques  for  a  multi-stage 
rocket  with  different  construction  parameters  and 
propellent  specific  impulses  in  each  stage.  We 

wish  to  optimize  the  configuration  insofar  as  perform¬ 
ance  is  affected  by  changes  in  the  number  of  stages, 
and  redistribution  of  the  fuel  and  structure  weight 
among  the  stages.  An  optimization  criterion  might 
be  maximum  burnout  velocity  for  given  take-off  weight. 

6.  Seek  an  optimum  nozzle  area  ratio  for  rockets  opera¬ 
ting  in  a  vacuum  which  will  provide  maximum  perform¬ 
ance  for  the  stage  in  question.  (Here  the  specific 
impulse  of  the  propellent  increases  monotonically 
with  increasing  ratio  of  exit  area  to  throat  area,  but 
the  increased  weight  of  larger  nozzles  degrades  the 
performance . ) 


VIII.  OPTIMUM  EFFORT  OR  SEARCH 


Suppose  that  an  object  is  somewhere  in  a  given  area,  and 
that  its  probabilities  of  being  in  the  various  possible  posi¬ 
tions  are  known. 

Suppose,  further,  that  a  limited  total  amount  of  search¬ 
ing  effort  (or  time)  is  available. 

Finally,  assuming  that  the  law  of  detection  is  known, 
the  chance  of  finding  the  object  when  a  given  amount  of  search 
is  carried  out  in  its  vicinity  is  determined. 


The  major  problem  is  to  find  the  optimum  manner  of  distri¬ 
buting  the  available  searching  effort:  the  one  which  maximizes 
the  chance  of  finding  the  object  (i.e.,  remember  detection  is 
not  certain.  Rather,  detection  is  an  event  which  may  have  any 
probability  between  zero  and  one  associated  with  it) . 

If  the  object  is  equally  likely  to  be  anywhere  within  a 
certain  area,  the  problem  is  straightforward.  The  search 
effort  is  evenly  laid  out  over  as  much  of  the  area  as  we  can 
search . 

But  if  the  chance  that  the  "enemy"  is  present  varies  from 
area  to  area,  the  problem  can  become  quite  difficult.  As  an 
example,  if  the  enemy  is  twice  as  likely  to  be  in  one  area  than 
another,  and  only  a  small  amount  of  search  area  is  available, 
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all  this  effort  should  be  spent  in  searching  the  more  likely 
area . 


Additional  by-products  are  such  questions  as  "What  would 
be  a  good  course  of  action?  Or  several  courses,  possibly?" 
"What  is  the  best  way  for  the  opponent  to  hide?" 

A  more  specialized  problem  is  as  follows.  A  target  is 
known  to  be  in  a  large  volume.  Using  a  pulse  radar,  the  volume 
is  searched  by  scanning  systematically  over  the  entire  space. 
Assume  further  that  the  search  situation  can  be  described  by 


1 .  the  size  of  the  volume  the  target  is  known  to 
be  located  in; 

2.  the  strength  of  the  return  signal. 


The  parameters  to  be  optimized  are; 


1.  the  return  threshold; 

2.  the  size  of  the  unit  search  interval; 

3.  The  time  spent  examining  each  interval. 


A  related  problem  is  the  optimum  acquisition  procedure, 
i.e.,  to  minimize  the  average  time  to  acquire  a  target  with  a 
radar  while  constraining  the  cumulative  probability  of  a  false 
alarm. 


IX.  SETTING  THE  OPTIMIZATION  PROBLEM 


I f  we  now  consider  systems  primarily,  henceforth,  we  can 
list  five  conditions  which  largely  determine  any  problem  in 
system  optimization: 


1.  Purpose  of  the  system; 

2.  Nature  of  the  inputs; 

3.  Criterion  of  goodness  of  performance  to  be  used; 
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4.  Freedom  of  choice  to  be  allowed  in  design; 

5.  In  practical  problems,  cost  of  the  system  in  a 
generalized  sense  must  be  included. 


Whenever  these  five  conditions  are  specified,  some  kind 
of  optimization  problem  is  defined,  although  it  may  be  such 
that  the  problem  has  no  solution  at  all,  or  no  best  solution, 
or  no  unique  best  solution. 

The  question  as  to  whether  an  optimum  exists  or  not  is 
dependent  in  general  on  the  existence  of  at  least  two  opposing 
functions  in  the  system.  For  example,  in  the  determination 
of  economic  lot  size,  the  total  variable  costs  are  the  sum  of 
carrying  charges  plus  preparation  costs,  as  shown  below: 


The  most  economic  lot  size  occurs  where  the  total 
variable  cost  is  a  minimum. 

Another  example:  The  velocity  required  to  leave  a  satel¬ 
lite  orbit  about  one  planet  and  proceed  to  another  planet  de¬ 
pends  upon  the  radius  of  the  satellite  orbit.  The  larger  the 
satellite  orbit,  the  weaker  is  the  gravitational  attraction 
of  the  home  planet.  This  has  a  tendency  to  reduce  the  velocity 
required  to  go  to  another  planet. 

But  the  larger  the  satellite  orbit,  the  lower  is  the  cir¬ 
cular  velocity  of  an  object  in  that  orbit.  This  has  a  tendency 
to  increase  the  additional  velocity  required.  Therefore,  there 
is  an  optimum  satellite  orbit  of  departure  of  arrival  to  go 
from  each  planet  to  each  other  planet. 
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On  the  other  hand,  if  no  opposing  functions  exist,  there 
may  be  no  optimum. 


However,  the  presence  of  bounds  on  the  variables  as  constraints 
may  serve  to  introduce  the  so-called  end-point  or  boundary 
optimum,  as  mentioned  earlier. 


In  the  above  case,  we  see  a  minimum  occurs  on  the  boundary  at 
x  *  a  . 

Other  nonmathematical  difficulties  may  beset  us.  Suppose 
we  have  two  given  functions  u(x,  y,  z  ...)  and  v(x,  y,  z  ...) 
to  make  as  large  as  possible  (e.g.,  quality  and  profit),  but  we 
are  thwarted  by  the  fact  that  increasing  one  of  them  decreases 
the  other. 
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Two  traditional  methods  are  available; 

1.  Fix  one  variable,  say  u,  at  that  particular  value 

u  the  least  that  will  be  tolerated,  and  maximize 
o  ’ 

v  subject  to  this  constraint. 

2.  Take  a  weighted  average  of  the  two  variables 


au  +  bv 


where  a  and  b  are  non-negative  and  sum  to  unity,  and  minimize 
this  mean,  to  give  an  optimum  u^ ,  v^. 

In  general,  the  two  optima  at  u  and  u.  will  not 
coincide. 


The  point  is,  the  choice  of  either  u  ,  (the  least 
tolerable  u)  or  the  weights  a,  b  (the  balance  between  u  and  v) 
is  not  a  mathematical  but  an  executive  decision. 
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X.  OPTIMIZATION  OF  LINEAR  CLOSED  LOOP  SYSTEMS 


A  fundamental  function  of  feedback  controlled  systems 
is  to  maintain  certain  variables  (called  errors)  at  constant 
or  minimum  values,  under  a  particular  set  of  conditions  which 
include : 


1.  dynamical  nature  of  the  process  to  be  controlled; 

2.  difficulties  in  accurate  measurements; 

3.  servo  limitations  as  to  power  and  saturation; 

4.  random  disturbances  and  inputs; 

5.  noise. 


Any  effort  to  establish  a  routine  design  procedure  of 
such  a  system  requires  specific  criteria  defining  the  optimum 
system. 

We  are  fortunate.  There  is  no  dearth  of  criteria.  Among 
the  multitude  are  such  ones  as  zero  error,  response  specified 
by  a  model,  variable  damping  dependent  on  error  size,  minimum 
lead  and  bandwidth,  transient  response  characteristics,  optimum 
impulse  response,  minimization  of  the  effects  of  disturbances, 
maximization  of  system  output,  minimization  of  the  square  of 
weighted  noise  and  dynamic  error,  maximization  of  load  torque 
at  constant  speed,  stable  equilibrium,  any  of  a  host  of  integral- 
of-the-error  type  of  criteria,  including  minimum  mean  square 
error . 


This  great  number  of  various  criteria  can  be  reduced  in 
general  to  three  basic  sets  of  performance  criteria,  existing 
at  present  in  problems  of  servo  mechanism  design.  They  are: 


1.  Stability; 

2.  Steady-state  conditions; 

3.  Transient  response  characteristics. 
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In  general,  a  device  which  is  not  stable  will  not  be 
used.  (In  some  multiple  loop  feedback  systems,  sometimes  an 
inner  loop  by  itself  might  be  deliberately  designed  to  be  un¬ 
stable  in  order  to  achieve  some  desired  benefit,  but  when  the 
subloop  is  embedded  properly,  the  over-all  multiple  loop  sys¬ 
tem  must  be  stable.) 

Transient  response  refers  to  the  manner  in  which  the  de¬ 
vice  arrives  at  a  steady  state.  Investigation  of  the  system 
for  the  case  of  transient  (or  nonsteady-state)  phenomena 
usually,  but  not  always,  requires  approaching  the  problem  in 
the  time  domain. 

Optimum  transient  response  may  be  defined  as  that  response 
to  a  step  input  which  regains  position  correspondence  in  minimum 
time  and  with  prescribed  limitations  on  overshoot  [e.g.,  for  a 
nonlinear  relay  servo  we  might  prescribe  no  overshoot  at  all] . 

Other  definitions  also  exist. 

j 

The  phrase  "optimum  adjustment"  signifies  the  problem  of 
setting  the  adjustable  parameters  of  a  control  loop  so  that 
the  control  action  resulting  after  a  disturbance  will  take 
place  in  the  best  possible  manner. 

Of  course,  no  such  optimum  adjustment  will  be  universally 
applicable,  because  it  is  always  based  upon  the  criterion  used 
to  define  optimum  control  action.  The  choice  of  such  a  cri¬ 
terion  is  rather  subjective  and  depends  upon  each  application. 

i  i 

i  i 

i  ! 

Steady-state  conditions  refer  to  the  error  (between  input 
and  response)  which  remains  after  all  transients  die  out. 

I 

Determination  of  the  optimum  transfer  function  in  the 
steady  state  requires  working  in  the  frequency  domain. 

I 

There  are  two  distinct  methods  of  design  which  we  will 
encounter  now  in  automatic  feedback  control  and  later  in  optimal 
filtering.  They  are: 


1.  Parameter-optimized,  or  fixed  configuration,  or 
relative  optimum. 

2.  Variation-optimized,  or  f ree-conf iguration,  or 
absolute  optimum. 

r 

4 
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Parameter -Opt imized - This  is  a  simplified  method  in 

which  it  is  necessary  at  the  outset  to  decide  the  interconnec¬ 
tion  of  the  elements  or  blocks  to  be  employed.  The  procedure 
to  optimize  parameters  in  the  proposed  (or  existing)  system 
consists  in  varying  the  parameter  values  that  are  not  speci¬ 
fied  previous  until  the  system  gives  minimum  mean-square 
error,  i.e.,  after  first  obtaining  an  expression  fcr  the  mean- 
squared  error  in  terms  of  the  parameters  of  the  system,  the 
optimum  parameter  values  may  be  determined,  using  ordinary 
minimizing  techniques  of  calculus. 

The  success  depends  to  some  extent  on  the  wisdom  of  the 
initial  choice  of  interconnections  and  elements.  There  must 
always  remain  some  doubt  as  to  whether  some  other  intercon 
nection  might  not  lead  to  a  better  result. 


Variat ion-Qpt imized  --  This  method  relieves  the  designer 
completely  of  the  onus  of  an  initial  choice  of  elements.  It 
is  based  upon  the  calculus  of  variations  and  when  the  other 
fixed  elements  in  the  system  have  been  specified,  it  allows 
no  choice  in  the  remaining  elements,  but  gives  directly  an  ab¬ 
solute  optimum  which  cannot  by  any  linear  means  be  improved 
upon . 


Such  systems  may  be  difficult  to  realize  in  a  practical 
form  since  the  method  takes  no  account  of  practical  convenience, 
but  the  existence  is  of  great  value  as  a  standard  for  compari¬ 
son  with  the  more  conventional  type. 

A  criterion  for  use  in  the  optimization  of  a  closed  loop 
system  may  be  formulated  as  follows; 

Let  z(t)  be  the  desired  system  output,  and  let  c(t)  be 
the  actual  system  output.  Then  any  functional  of  c(t)  and 
z(t)  is  some  kind  of  measure  of  how  well  the  system  operates. 
Usually,  a  measure  of  system  performance  is  some  quantity 
which  depends  on  the  error 


z(t)  -  c ( t ) 


and  which  is  a  minimum  when  the  error  is  zero,  and  becomes 
larger  when  error  is  increased. 
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The  most  important  criterion  in  use  is  the  mean  square 
error  criterion 


/[ z(t)  -  c  ( t ) ] 2  dt 


which  is  used  mainly  because  it  permits  the  development  of 
analytic  methods  for  synthesizing  systems  with  random  inputs. 

For  many  situations,  other  criteria  are  more  suitable  but 
suffer  from  lack  of  mathematical  development.  This  criterion 
also  has  a  physical  interpretation  of  discriminating  against 
the  occurrence  of  large  errors,  i.e.,  the  optimization 
criterion  of  minimum  mean  square  ensemble  system  error  weights 
large  errors  more  heavily  than  small  errors.  It  reduces  the 
likelihood  of  large  errors  but  leaves  the  system  relatively 
sensitive  to  small  errors. 

However,  other  criterion  might  be  more  important  in 
some  cases.  For  example,  it  might  be  more  appropriate  to  maxi¬ 
mize  the  probability  that  the  error  be  less  than  some  pre¬ 
scribed  tolerance, 


Prob  {J  Z(t)  -  c  (t)  |  <  K)} 

i.e.,  we  require  a  system  which  minimizes  the  specified  proba¬ 
bility.  All  errors  greater  than  a  certain  threshold  are 
equally  bad,  while  small  errors  are  tolerated. 

Prob  £|z(t)  =  c (t )  |  s(  t),  t  <^t} 


i.e.  we  require  the  system  whose  output  has  the  largest  con¬ 
ditional  probability,  using  all  the  past  history  of  the  sig¬ 
nal,  of  being  the  correct  value.  But  this  requires  continuous 
conditional  probabilities;  assumes  all  errors  larger  than  a 
certain  value  are  equally  bad;  and  requires  a  complete  sta¬ 
tistical  knowledge  of  inputs,  often  not  available. 


XI .  OPTIMAL  FILTERING  AND  PREDICTION 


Conventional  frequency  filters  are  intended  to  separate 
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two  classes  of  signals  whose  spectra  do  not  overlap.  Specifi¬ 
cation  of  these  filters  does  not  depend  on  the  statistical 
properties  of  the  signals. 

However,  communication  and  control  systems  often  must 
perform  the  task  of  separating  as  well  as  possible  a  desired 
signal  input  from  an  extraneous  signal  input  such  as  random 
noise,  whose  spectrum  greatly  overlaps  that  of  the  desired 
signal.  Optimum  filters  (relative  to  the  way  they  are  speci¬ 
fied)  are  a  class  of  filters  designed  to  perform  this  separa- 
t  ion . 


Assume  we  have  a  corrupted  signal  y(t)  which  is  the  sum 
of  a  desired  signal  s(t)  and  unwanted  noise  n(t) 


y ( t )  -  s(t)  +  n(t)  . 


Smoothing  is  the  removal  of  the  unwanted  random  roughness 
in  the  data".  Tn  some  cases  n(t)  has  higher  frequency  com¬ 
ponents  than  s(t),  and  removal  of  n(t)  actually  amounts  to 
smoothing  the  graph  of  the  signal. 

Predicting  is  the  forecasting  of  a  future  value  of  the 
desired  input  signal . 

Smoothing  and  predicting  can  be  combined  together,  as 
well  as  with  and  without  differentiation. 

Two  fundamental  principles  of  smoothing  and  predicting 

are : 

1.  No  separation  of  signal  s(t)  from  s(t)  +  n(t)  is 
possible  unless  s(t)  and  n(t)  have  distinguishing 
properties . 

2.  No  prediction  of  s(t)  into  the  future  can  be  made 
unless  it  has  known  property  which  relates  its  past 
and  future,  at  least  in  some  statistical  sense. 


The  term  "filter"  or  "memory  function"  used  hereafter 
is  to  be  taken  in  a  very  broad  context.  Although  mathema¬ 
tically  it  may  be  termed  an  "operator,"  physically  in  this 
sense  it  may  be  a  suitable  electric  circuit  or  a  more  compli 
cated  piece  of  equipment,  such  as  an  automatic  feedback  con¬ 
trol  loop;  or  a  memory  store  containing  a  given  set  of 
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transmitted  messages  for  comparison  with  a  garbled  message 
received;  or  various  electronic  circuits,  or  even  a  missile 
itself . 

Consider  the  general  system 


y(t)  =  s(t)  +  n(t) 


Input 


a 


FILTER 


c(t)  -  s(t  +  T 


Output 


A  smoothing  filter  is  designed  to  extract  as  well  as 
possible  a  desired  signal  s(t)  from  the  mixture  y(t)  of 
signal  and  noise.  Here  T  *  0,  and  the  system  is  also  called 
a  "duplicator." 

A  predicting  filter  is  designed  to  yield  a  future  value 
s(t  +  T)  ,  T>0  of  thesignal,  where  the  signal  s(t)  may  or 
may  not  (n(t)  /  0)  be  mixed  with  noise  in  the  present. 

The  over-all  problem  of  designing  systems  to  perform 
smoothing  and  predicting  can  be  considered  in  two  parts: 


1.  Determine  by  some  criterion  what  the  best  smoothing 
and  predicting  is,  and  what  the  optimum  performance 
operator  is. 

2.  Realization  of  the  desired  performance  operator  in 
a  workable  device. 


Until  the  work  of  Wiener  became  known,  the  design  of 
linear  systems  depended  on  a  combination  of  cut-and-try  pro¬ 
cedures  and  analytical  methods  for  choosing,  in  some  optimum 
fashion,  the  free  parameters  of  a  system  of  given  form. 

Norbert  Wiener  (1942)  solved  the  problem  of  optimum  prediction 
and  filter  under  these  four  assumptions: 


1.  The  system  is  a  fixed  parameter-linear  device  (since 
extended  to  a  time-varying  parameter  linear  device)  . 

2.  The  system  has  infinite  memory,  i.e.,  operates  on 
all  past  history  of  the  signal  and  noise  (since  ex¬ 
tended  to  include  the  cases  of  finite  duration 
sampling  time) . 
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3.  The  input  time  series  (both  signal  and  noise)  are 
ergodic  stationary  random  processes  (since  extended 
to  the  case  of  nonstationary  nonergodic  random 
process  input)  . 

4 .  The  system  is  optimum  in  the  sense  of  minimization 
of  least  squares  (this  has  not  been  generalized  in 
a  practical  manner)  . 


The  Wiener  method  of  system  optimization  specifies 
mathematically  that  memory  function  which  makes  output,  c(t), 
the  "best"  approximation  to  a  translation  of  the  message 
input  s(t  +  t )  where  T  is  some  real  number. 


y  (t) 


-  s(t)  + 
Input 


n(t ) 


W(t) 
System 
Memory 
Funct ion 


c(t) 

Output 


The  error  in  the  approximation,  t(t),  is  defined  as 

e  (t)  =  s(t)  -  c ( t ) 


and  the  approximations  will  be  assumed  "best"  when  the  mean 
square  of  £(t)  is  minimum,  i.e., 


e 


2 


-  Lim  2Y 
T  ->oo 


[e  (t ) ] 2  dt 


The  mathematical  application  of  the  Wiener  method  is  as 
follows : 


1 .  Express  the  mean  squared  error  in  terms  of 
statistical  properties  of  the  message  and  noise 
s igna Is . 

2.  Minimize  the  mean  squared  error  by  use  of  the  cal¬ 
culus  of  variations.  In  the  process  of  minimization, 
it  will  be  found  that  the  optimum  memory  function 
must  satisfy  a  Wiener-Hopf  type  integral  equation. 
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3.  The  desired  optimum  memory  function  will  be  expressed 
in  terms  of  the  solution  to  the  integral  equation 
and  characterized  by  transfer  function  relating  the 
output  to  the  input. 

(Memory  function  is  to  be  regarded  in  the  broad  context 
as  a  component,  subsystem,  or  even  a  system,  depending 
on  the  problem.) 

Some  important  limitations  on  the  Wiener  method  which 
must  be  kept  in  mind  are: 


1.  All  physical  systems  to  which  this  method  is  applied 
are  linear. 


2.  In  many  cases,  it  is  impossible  to  realize  practi 
cally  the  memory  function  derived  analytically  by 
the  Wiener  method.  The  design  is  feasible  provided 
the  optimum  process  does  not  require  a  circuit  to 
distinguish  between  positive  and  negative  frequen¬ 
cies,  and  does  not  require  it  to  have  a  negative 
memory.  (A  negative  memory  would  imply  that  a  sig¬ 
nal  could  be  processed  before  it  had  been  received. 

3.  The  auto  and  cross-correlation  functions  and 
spectral  densities  functions  for  the  noise  must 
exist. 


A  relevant  question  is  "Why  bother  with  optimum  theory, 
when  the  devices  may  turn  out  to  be  physically  unrealizable?" 
Two  reasons  exist. 


1.  It  is  desirable  to  have  a  standard  of  reference  re¬ 
presenting  the  maximum  attainable  performance  that 
can  be  expected  for  an  optimum  system.  By  consider¬ 
ing  the  theoretical  performance  capabilities  of 
optimum  systems,  one  can  often  show  that  a  simple, 
practical  system  under  study  may  differ  so  little 
from  optimum  as  to  make  further  refinement  unneces¬ 
sary  . 

2.  Even  though  optimum  filters  may  not  of  themselves  be 
easily  instrumented,  they  can  often  be  approximated 
by  practical  devices. 
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The  Wiener  method  yields  an  "absolute"  optimum  system 
memory  function,  whereas  the  so-called  Phillip's  method  de¬ 
rives  a  "relative"  optimum  system  memory  function. 

This  relative  optimum  is  found  by  assuming  a  basic 
structure  for  a  system  and  optimizing  with  respect  to  its 
controllable  parameter. 

Note  that  the  matter  of  physical  reality,  a  severe  limi¬ 
tation  of  the  Wiener  method,  does  not  enter  and  no  consider¬ 
ation  is  given  whether  or  not  the  opt imum  system  could  be 
practically  realized.  Since  the  basic  structure  is  fixed  prior 
to  optimization,  the  question  does  not  arise.  An  optimum 
memory  function  is  then  derived  as  follows: 


1.  Establish  an  expression  for  the  mean-squared  error 
of  the  system; 

2.  Minimize  this  error  with  respect  to  controllable 
parameters . 


For  example,  the  realizable  linear-over-quadratic  trans¬ 
fer  function 


as  +  b 

— 2 - - - 

cs  +  ds  +  e 


as  a  beam  riding  computer  filter  for  a  guided  missile  can  be 
shown  to  approximate  the  results  of  the  theoretical  optimum 
filter  for  the  case  of  "white"  noise  (all  frequencies  repre¬ 
sented)  and  game  theory  acceleration  spectra  (discussed  later). 

It  is  unfortunate  that  guidance  filtering  which  is 
optimal  from  the  standpoint  of  minimizing  the  maximum  mean 
square  miss  distance  (or  beam  riding  error)  demands  infinite 
mean  square  missile  acceleration.  It  seems  feasible  to  take 
into  account  the  actual  physical  limiting  of  missile  accelera¬ 
tion  by  seeking  the  filter  which  minimizes  the  mean  square 
miss  (or  beam  riding  error)  subject  to  a  constraint  on  the 
allowed  mean  square  missile  acceleration. 

However,  by  separating  the  guidance  and  control  functions 
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in  the  missile,  we  can  include  the  effects  of  target  maneuver 
in  the  optimization  of  the  beam  riding  computer  with  drag-in¬ 
duced  slowdown  as  a  constraint,  and  include  the  effects  of 
noise  in  the  optimization  of  the  autopilot  with  acceleration 
limiting  as  the  constraint  occurring  there. 

Eventually,  we  define  the  optimum  beam  riding  computer 
to  be  that  one  which  minimizes  total  beam  riding  errors  due 
to  both  noise  and  target  maneuvers  while  restricting  the  in¬ 
duced  drag  due  to  noise  to  some  acceptable  value,  i.e.,  an 
acceptable  value  of  induced  drag  is  that  value  which  does  not 
cause  the  missile  to  slow  down  in  level  flight  at  the  maximum 
intended  range. 

The  near-optimum  transfer  function  is  no  longer  linear- 
over-quadratic,  but  becomes  linear  over-quartic,  due  to  the 
constraints . 

As  a  side  note,  we  should  mention  that  while  optimization 
of  filters  concerned  with  smoothing  and  predicting  data  has, 
on  the  whole,  been  done  with  respect  to  the  mean  square  error 
criterion,  the  filters  are  sometimes  complicated  to  compute, 
making  them  quite  often  unsuited  for  real  time  solutions.  For 
such  applications,  it  may  be  necessary  to  optimize  the  filters 
from  the  primary  viewpoint  of  ease  of  computation,  with  which 
the  final  estimates  of  the  output  are  obtained,  and  optimiza¬ 
tion  in  the  mean  square  value  sense  is  obtained  as  a  secondary 
consideration  in  order  to  provide  some  control  of  the  mean 
square  output  error.  This  approach  would  have  the  greatest 
appeal  for  problems  involving  real  time  filtering  where  com¬ 
puting  time  and  complexity  are  primary  considerations,  and 
noise  reduction  is  a  secondary  consideration. 

A  remark  on  the  use  of  nonlinear  filters  can  be  made 
here.  If  the  noise  and  signal  at  the  input  both  possess 
Gaussian  distributions  in  amplitude,  the  linear  filter  is  the 
optimum  filter,  and  no  improvement  in  filtering  can  be  realized 
by  going  to  a  nonlinear  device.  But  with  more  general  input 
signals,  the  same  situation  does  not  hold.  In  many  cases  the 
mean  square  error  can  be  further  reduced  by  the  addition  of 
nonlinear  filters. 

In  concluding  this  section,  we  can  state  a  few  facts 
about  detection  of  signals  in  noise.  Detection  is  the  process 
of  determining  whether  a  signal  ii.  present  or  not.  When  de¬ 
tecting  a  signal,  the  detector  is  either  right  or  wrong,  but 
in  the  theory  of  prediction,  it  is  unlikely  that  one  is  ever 
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exactly  right,  but  there  are  all  degrees  of  wrongness.  In  de 
tection,  alternative  optimization  criteria  to  minimization  in 
a  least  squares  sense  has  been  used.  For  example; 


1.  Maximization  of  signal-to-noise  ratio  at  a  speci¬ 
fied  instant  of  time; 

2.  Maximization  of  the  absolute  magnitude  of  the  differ¬ 
ences  between  signal  and  noise  over  all  time; 

3.  In  one  optimum  threshold  (i.e.,  weak  signal)  detec¬ 
tion  system,  optimality  is  achieved  by  minimizing 
the  average  cost  of  decision; 

4 .  Another  detection  system  is  "best"  which  in  the  long 
run  will  hold  fixed  the  false  alarm  probability  and 
will  minimize  the  probability  of  missing  the  signal. 


XII.  WIENER  THEORY  APPLIED  TO  AUTOPILOT  DESIGN 


The  Wiener  optimum  linear  filter  formulation  is  applied 
to  the  autopilot  closed  loop  transfer  function  with  the  dis¬ 
turbance  spectrum  entering  at  the  wing.  [Actually  the  noise 
in  the  autopilot  enters  the  loop  at  various  places.  ] 

A  statistically  optimum  autopilot  may  be  defined  as  that 
autopilot  which  minimizes  the  mean-square  error  between  accelera¬ 
tion  command  and  the  missile  acceleration  response  attained. 

For  a  realistic  approach,  saturation  of  various  elements 
in  the  system  must  be  included.  Because  of  the  nonlinear 
nature  of  these  saturating  elements,  general  procedures  for 
the  optimization  of  filters  involving  them  have  not  been  de¬ 
veloped.  Usually  the  nonlinear  limits  are  replaced  by  a  linear 
system  by  constraining  the  saturating  quantities  to  their  mean 
square  values. 

It  is  desired  to^find  the  transfer  function  G  which  mini¬ 
mizes  the  mean  square  e  ,  subject  to  a  constraint  of  limited 

~2 

mean  square  wing  rate  6  . 
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Disturbance 


Accelerometer 


XIII.  GAME  THEORY  AND  OPTIMAL  FILTERING 


The  theory  of  games,  or  game  theory,  is  the  particular 
branch  of  applied  mathematics  which  deals  with  the  rational 
analysis  of  competitive  and  cooperative  systems,  and  the  de¬ 
termination  of  optimal  courses  of  action  for  the  participants. 
Situations  to  which  game  theory  can  be  applied  occur  in  opera¬ 
tions  analysis,  strategic  and  tactical  planning,  whenever 
there  is  conflict  of  interest  arising  from  the  actions  of  an 
opponent.  ("Opponent"  can  include  even  Nature,  a  fictitious 
player,  having  no  known  objective  nor  strategy  in  some  general 
cases . ) 

The  elementary  concepts  for  the  simplest,  two-person 
game  are  that  which  one  side  gains  the  other  side  loses,  that 
the  opponents  simultaneously  choose  a  course  of  action  (called 
a  strategy)}  and  that  the  outcome  of  the  game  (the  payment  of 
one  side  to  the  other)  is  determined  by  this  dual  choice  of 
strategies.  The  outcome  is  not  completely  determined  by  either 
side  alone,  it  is  determined  by  the  combined  decisions  of  the 
two  opposed  players. 

One  of  several  possible  criteria  is  the  minimax  principle 
—  each  player  should  employ  only  optimal  strategies  which 
minimize  his  maximum  loss,  no  matter  what  the  other  player  may 
do . 
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In  Wiener’s  work,  the  statistical  properties  of  the 
noise  and  signal  are  assumed  fixed,  and  given  in  advance  of 
the  problem.  A  guidance  filter  can  be  optimized  then  to  in¬ 
clude  the  effects  of  target  random  maneuvers,  presumably 
typified  by  maneuvers  over  which  the  enemy  himself  has 
little  or  no  control  [e.g.,  wind  gusts,  noise  in  his  system, 
etc.].  Such  a  guidance  filter  or  beam  riding  computer  may  or 
may  not  be  near  optimum  if  the  target  maneuvers  in  a  deliber¬ 
ate  manner.  Essentially,  the  game  theory  approach  assumes  an 
intell igent  target,  one  able  to  assume  the  most  evasive 
maneuvers  within  its  capabilities.  The  goal  is  to  design  the 
best  possible  system  against  this  crafty  target,  within  cer¬ 
tain  boundary  conditions  and  constraints. 

In  game  theory  optimization,  allowance  is  made  for  the 
fact  that  the  enemy  who  is  producing  the  signal  may  prefer 
not  to  be  followed,  and  may  attempt  to  keep  the  mean-square 
error  as  large  as  possible.  The  spectral  density  of  his 
signal  is  thus  no  longer  fixed,  but  is  the  strategy  of  one  of 
the  participants  of  the  game. 

The  strategy  of  the  other  participant,  the  filter  de¬ 
signer,  is  specified  by  the  transfer  function  of  the  filter. 
The  game  payoff  is  considered  to  be  the  mean-square  error  or 
miss . 


The  filter  designer  must  design  the  best  filter  in  view 
of  the  worst  choice  of  signal  spectral  density  by  the  enemy 
(as  caused  by  his  maneuver) . 

The  enemy,  who  is  the  signal  producer,  must  generate  a 
signal  which  has  statistical  characteristics  that  result  in 
the  greatest,  possible  miss  despite  the  best  efforts  of  the 
filter  designer. 

If  these  two  considerations  are  mutually  compatible, 
there  is  a  game  theory  solution,  i.e,,  there  is  a  filter 
transfer  function  which  gives  the  smallest  mean-square  error 
for  the  worst  possible  signal. 


The  game  theory  solution  is  essentially  the  intersection 
of  two  functional  equations:  one  giving  tne  optimum  filter 
transfer  function  for  fixed  signal  spectral  density;  the  other 
the  optimum  signal  spectral  density  for  fixed  filter  function. 
This  solution  has  the  property  that  the  error  obtained  with  it 
when  the  target,  signal  strategy  is  optimum  (in  the  sense  of 
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maximizing  the  miss)  is  smaller  than  the  error  obtained  for 
any  other  filter  transfer  function  when  the  signal  strategy 
is  optimum  for  that  transfer  function. 


XIV.  WHY  OPTIMIZATION  TECHNIQUES  HAVE  NOT  BEEN  USED 

-  MORE  FREQUENTLY - 


Three  conditions  have  conspired  to  keep  optimization 
methods  from  widespread  application  in  the  past. 


1.  Complicated  mathematics,  for  example,  in  optimizing 
a  filter  by  Wiener's  method,  and  actual  nonlinear 
constraints . 

2.  Equipment  required  is  prohibitive  in  cost  and/or 
bulk. 

3.  Advantages  accruing  from  successive  refinements  be¬ 
come  progressively  smaller.  Quite  often  a  com¬ 
paratively  simple  filter,  say  roughly  approximating 
the  Wiener  design,  in  practice  gives  most  of  the 
advantage  to  be  expected  from  the  true  Wiener  design. 


XV.  RELATION  TO  ADAPTIVE  SYSTEMS 


Designing  control  systems  where  (a)  little  significant 
information  is  known  about  the  process  to  be  controlled,  (b) 
the  properties  of  the  process  vary  over  an  extraordinarily 
large  range,  and  (c)  the  characteristics  of  the  system  input 
signals  change  markedly  with  time,  may  require  systems  in 
which  the  compensation  is  automatically  adjusted  to  offset 
these  adverse  effects.  Such  systems  are  called  "adaptive 
systems .  " 

Optimum  performance  from  operating  systems  is  of  in¬ 
creasing  importance  as  competition  in  the  real  world  becomes 
more  severe.  Adaptive  control  is  a  method  of  automatic  con¬ 
trol  aimed  at  obtaining  optimum  system  performance  even  when 
there  exists  incomplete  or  inexact  analytical  or  analog  models 
of  the  process  that  is  being  controlled. 

Adaptive  or  self-optimizing  systems  optimize  control  by 
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the  use  of  an  automatic  unit  which  searches  out  and  holds  the 
best  performance  from  a  controlled  system,  in  spite  of  any 
reasonable  change  of  the  output  level  or  environmental  opera¬ 
ting  conditions.  In  this  sense,  the  adaptive  system  "learns" 
to  improve  its  performance,  based  upon  experience,  thereby 
adapting  itself  to  the  circumstances  it  finds. 

In  order  for  an  optimizing  action  to  be  possible,  the 
system  to  be  controlled  must  have  a  performance  characteris¬ 
tic  which  shows  an  optimum  point  as  one  or  more  of  the  inputs 
vary.  A  suitable  deviation  signal  that  represents  the  depar¬ 
ture  of  the  operation  from  the  optimum  condition  is  generated 
and  utilized  is  a  basis  for  making  corrections  to  the  input 
(if  controllable)  or  to  the  adaptive  system  parameters  (e.g., 
variable  damping  if  noise  is  the  input),  so  that  the  devia¬ 
tion  diminishes  and  the  operating  point  approaches  the  optimum 
point  again. 

An  application  of  adaptive  systems  is  made  in  target 
tracking  radars,  which  adjust  the  over-all  transfer  function 
of  the  tracking  loop  according  to  target  behavior  and  system 
noise.  Target  acceleration  or  maneuver  is  "recognized"  and 
the  system  parameters  vary  to  convert  the  function  of  the 
loop  from  data  smoothing  to  tight  and  fast  follow-up.  When 
target  maneuver  ceases,  system  returns  to  its  filtering  mode. 
Thus,  the  parameters  are  adjusted  to  balance  the  low  frequency 
error  (lag  due  to  maneuver)  and  the  high  frequency  error  (re¬ 
sponse  due  to  noise) . 


XVI.  MATHEMATICAL  PROGRAMMING 


Linear  programming,  nonlinear  programming  and  dynamic 
programming  are  three  of  a  set  of  techniques  called  "Mathe¬ 
matical  Programming"  which  involve  the  programming  of  inter¬ 
dependent  activities.  These  techniques  are  referred  to  as 
'programming"  in  order  to  emphasize  that  planning,  as  dis¬ 
tinguished  from  operations  or  execution  of  plans,  is  the  area 
of  primary  interest.  In  brief,  programming  is  concerned  with 
the  problem  of  planning  a  complex  of  interdependent  activities 
in  the  best  possible  way. 

The  general  programming  problem  is  to  maximize  or  mini¬ 
mize  an  objective  function  0(x),  which  is  some  over-all 
measure  such  as  cost,  or  profit,  or  value,  or  quality,  or 
efficiency,  etc.  subject  to  constraints  g^(x)  >  0,  gg(x)  rl 

.  .  .  ,  and  Xj  >  0 ,  x^  >  0 ,  ... 
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The  usual  analytic  methods  of  solving  extremization  prob¬ 
lems  in  the  presence  of  constraint  equations  (solving  the  con¬ 
straint  equations,  substituting  in  the  objective  function, 
and  differentiating,  or  else  by  Lagrange  multiplier  techniques) 
do  not  take  into  account  the  inequality  constraints  which 
characterize  mathematical  programming. 


XVII.  LINEAR  PROGRAMMING 


Linear  programming  : s  a  relatively  new  mathematical  tech¬ 
nique  to  handle  problems  with  the  following  characteristics. 


1.  There  is  usually  a  large  but  finite  number  of  non¬ 
negative  variables. 

2.  The  variables  are  subject  to  a  finite  number  of  con¬ 
straints  or  boundary  conditions  usually  in  the  form 
of  linear  inequalities  and/or  equations  which  limit 
the  variables  range,  and  are  accurately  known. 

3.  Under  these  constraints,  some  objective  function  is 
to  be  maximized  or  minimized. 


Both  the  objective  function  to  be  maximized  and  the  re¬ 
strictions  on  each  variable  (equalities  or  inequalities)  are 
linear  in  the  variables. 

We  say  a  solution  is  feasible  if  it  satisfies  the  con¬ 
straints,  and  optimal  if  it  also  achieves  a  maximum.  The 
problem  consists  in  determining,  out  of  the  infinite  number  of 
feasible  solutions,  a  unique  (if  possible)  solution  which  is 
optimal . 

The  resulting  solution  will  then  provide  the  best  possible 
planning  of  operations  under  the  specified  restrictions. 

Problems  with  these  characteristics  crop  up  in  transporta¬ 
tion  fields  (relating  sources  and  destinations  of  supplies) , 
production  bottlenecks  (efficient  allocation  of  limited  re¬ 
sources)  problems  of  scheduling  and  timing,  contract  awards, 
personnel  assignment,  etc. 

Linear  programming  problems  have  been  attacked  in  several 
ways,  the  most  prominent  method  being  the  Simplex  Algorithm, 
which  is  a  very  powerful  computational  technique  which  can 
efficiently  solve  large  systems  containing  hundreds  of  equations. 
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However,  limitations  and  disadvantages  to  linear  pro¬ 
gramming  exist.  Among  them  are: 


1.  The  linearity  restriction.  For  example,  linear 
costs  do  not  penalize  large  values  of  the 
variable.  [This  has  been  alleviated  to  some  ex 
tent  by  the  generalization  to  "Quadratic  Program¬ 
ming,"  in  which  the  linear  objective  function  to 
be  extremized  can  be  replaced  by  a  quadratic  form, 
with  linear  constraints.] 

2.  Optimal  solutions  are  not  obtained  in  analytic  form. 
Changes  in  the  mathematical  model  require  recalcula- 
t  ion . 

3.  Error  analysis  is  difficult. 

4.  No  provision  is  made  for  relationships  involving 
uncertainty  due  to  random  fluctuations  or  errors 
in  determination,  for  example,  sales  forecast  in 
the  form  of  a  probability  distribution  cannot  be 
handled . 

5.  The  expression  of  realistic  objectives  and  constraints 
in  measurable  terms. 

6.  The  determination  of  suitable  numerical  values  for 
coefficients . 

7.  The  computational  labor  required  to  execute  numeri¬ 
cally  large-scale  linear  programming  problems. 


XVIII.  DYNAMIC  PROGRAMMING 


Among  programming  problems,  there  are  some  in  which  time 
plays  an  essential  role  and  in  which  the  sequence  of  decisions 
is  vital.  These  are  termed  "Dynamic  Programming"  problems,  and 
Dynamic  Programming,  the  functional  equation  technique  of  a 
new  mathematical  discipline,  can  be  used  in  the  formulation  and 
solution  of  optimization  problems,  including  those  in  which  the 
process  need  not  necessarily  change  with  time.  Furthermore, 
the  process  may  be  stochastic,  i.e.,  the  outcome  is  not  de¬ 
termined  but  is  predictable  by  means  of  a  probability  distribu- 
t  ion . 
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In  a  typical  application,  a  sequence  of  decisions  is 
sought  which  in  some  sense  optimizes  the  behavior  of  a  sytem. 

In  these  sequences  of  operations  the  outcomes  of  the  preceding 
operations  may  be  used  to  guide  the  course  of  future  operations. 
As  the  number  of  decisions  increases,  and  the  discrete  length 
of  the  decision  interval  decreases  without  limit,  continuous 
solutions  are  produced  which  are  equivalent  to  those  furnished 
by  the  classical  calculus  of  variations.  Thus,  Dynamic  Program¬ 
ming  is  properly  an  extension  of  the  calculus  of  variations, 
but  of  much  wider  scope  and  versatility. 


XIX.  QUEUEING  THEORY 


In  many  operations  there  is  a  lack  of  timing  between 
arrival,  at  some  point  in  the  operation,  of  a  sequence  of  units, 
and  the  subsequent  disposal  of  these  units,  so  that  a  waiting 
line  or  queue  is  formed  of  newly  arrived  units  awaiting  disposal. 
For  example,  the  units  might  be  aircraft  "stacked"  over  an  air¬ 
port,  traffic  tie-up,  ships  in  a  harbor  awaiting  docking, 
customers  in  a  cafeteria,  etc.  Queueing  Theory  or  Waiting  Line 
Theory  is  the  specialized  method  for  analysis  of  these  situa¬ 
tions  . 


A  central  problem  of  waiting  line  theory  is  the  relation¬ 
ship  between  the  mean  length  of  the  waiting  line  and  the  degree 
of  randomness  of  arrival  and  disposal,  such  a  line  arising 
whenever  the  mean  arrival  rate  exceeds  the  mean  service  rate. 

On  this  problem  can  be  based  estimates  of  the  optimum  capacity 
of  the  service  facilities  when  one  balances  the  cost  of  letting 
the  unit  wait  in  line  against  the  cost  of  increasing  the  service 
rate . 


XX.  THE  CONVERSE  OF  OPTIMAL 


The  converse  of  "optimal"  i.e.,  the  worst  of  the  worst 
has  been  dubbed  "pessimal"  by  J  L.  Vanderslice  (APL) .  However, 
this  tern,  is  not  in  common  usage  in  the  technical  journals. 
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I.  LINEAR  FILTERS,  THEIR  TRANSFER  FUNCTIONS 
AND  WEIGHTING  FUNCTIONS* 


Let  x(t)  and  y(t)  be  the  input  and  output,  respec¬ 
tively,  of  a  filter.  By  this  we  understand 
that  there  exists  a  linear  differential  equa¬ 
tion  with  constant  coefficients  connecting  x(t)_ 
x(t)  and  y(t).  We  shall  temporarily  assume 
this  equation  to  have  the  form 


1 

1  S 

jFILTER 

y(tr 

(1.1) 


.  (n)  h  (n-1) 

n  y  +  Dn-1  y 


+...+  b, y  +  bQy  =  an-1  x 


(  n-1 ) 


n-1 


+  .  .  .+  a.x  +  a  x 
i  o 


where 


(1.2)  bn  /  0. 


Replacing  the  operator  by  the  letter  S,  (some 
people  use  D,  others  P) ,  we  can  formally  solve  for  y,  thus: 


(1.3) 


y  = 


„  n  - 1  _ 

a„,S  + . . . +  a, S  +  a 
n-1 _  1 _ o _ 

b  Sn  +  b  .  Sn_1  +  . .+  b, S  +  b 
n  n-1  1  o 


x  =  F(S)  x 


The  rational  function  of  S,  F(S),  is  called  the 
transfer  f unction  of  the  filter.  In  elementary  differential 


*Some  of  the  discussion  of  this  and  subsequent  sections  is 
simplified,  and  not  completely  rigorous  mathematically.  The 
purpose  of  this  paper  is  to  acquaint  the  reader  with  useful 
concepts  and  techniques,  which  is  oftentimes  at  odds  with  the 
objectives  of  completeness  and  rigor. 
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equations  it  is  shown  that  if  one  solves  for  the  roots  of 

b  Sn  +  b  ,  Sn"l  +  ...+  b,S  +  b  =0,  and  if  the  roots  all 
n  n-1  1  o 

have  negative  real  parts,  then  the  equation  (1.1)  is  stable, 

i.e.,  if  x(t)  =  0,  then  y(t)  will  approach  zero  no  matter 

what  the  initial  values  of  y,  y  y^n_1^ .  Accordingly, 

if  the  denominator  of  F(S)  (extreme  caution  must  be  exercised 

in  cancelling  common  factors  of  numerator  and  denominator) 

has  the  ascribed  property,  we  shall  say  the  filter  is  stable 

or  realizable.  (The  term  "realizable"  is  found  in  the 

literature  but  its  use  in  this  connection  is  objectionable.) 

Let  the  filter  be  stable,  and  be  of  the  form  given 
by  (1.1).  The  following  statements  are  then  valid: 

A.  There  exists  a  unique  function  W( r) ,  called  the 
weighting  function  of  the  filter,  defined  for  x  ■>  0  such  that, 
for  any  sufficiently  well  behaved  input,  we  have  in  steady 
state 


oo 

y ( t )  =  J  W(x)  x  (t-x) 

o 


dx,  assuming  x(U) 


f iciently 
cases)  . 


defined  for  -  cd  <  U  <  t . 

r 

B.  F(S)  -J 


e  w(x)  dx 


large  real  parts.  (ReS  = 


for 
0  is 


complex  S  with  suf- 
large  enough  in  all 


C. 

D. 


F(S)  =  F(S),  F(S )  real  for  real  S. 


If  x(t)  =  A  cos  ut  ,  then  in  steady  state 


y(t)  =  A 


F(  iw) 


cos  (ut  +0),  where 


*  _  j.  Im  F(iu) 
tan  *  "  iieF(L) 


Justifications - Me  shall  conclude  this  section 

with  justifications  of  A,  B,  C,  D  and  a  short  discussion  of 
the  significance  of  them. 


A  is  a  constant. 


t 

iV 

i 
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Proof  c  f  A 

For  any  sufficiently  well  behaved  function  W(x),  we 

CD 

have,  setting  y(t)  =  f  W(t)  x  (t-r)  dT  . 


\JU 

(1.4)  bQy  =f  bQ  W(t)  x  (t-x)  dx  . 


VAy 

(1.5)  t>1y  =  j  bj^  W(x)  x  (t-x)  dx  =  bx  W(0)  x  (t) 


UJ 

/  bx  W(x) 


X  (t-T)  dT 


using  integration  by  parts.  Using  successive  integration  by 
parts,  for  k  <  n  we  have 


(1.6) 


bk  y 


(k) 


J  bk  W ( X )  X(k)  (t-x)  dT  =  bk  W(0)  X 


(k-1) 


(k-2) 


+  b.  W ( 0 )  xv~  w  (t)  +...+  b-W^  *'(0)  x(t) 


A“D 


+  f  bk  Wv  '(x)  x  (t-x)  dT 


Adding  up  the  above  n+1  equations,  there  results 


(1.7) 


b  v(n) 


yv  '  +  b 


v(n  1)  4.  +  h  v4"b^ 

n-1  y  1  y  o’ 
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(1.8) 


(1.9) 


CD 

=  f  [ bQ  W( t )  +  b,w( t)  +...+  bnW(n)  (  t)  ]  X  (t-r)  dT 
Jo 

+  bn  W(0)  x(n_1)  +  [bn  W(0)  +  bn_1  W(0)]  x(n_2) 

+...+  (bn  W(n_2)(0)  +...+  b3  W(0)  +  b2  W(0) ]  k 

+  [bn  w(n_1)  (o)  +...+  b2w(0)  +  bx  w(o) ]  X  . 

Note  that  if  we  can  satisfy 

bn  W(n)  (T)  +...+  bx  W ( x )  +  bQ  W( t )  =  0,  for  x  >  0, 
and 


b  W(0)  =  a 
n  n- 

b  W(0)  +  b  .  W(0)  =  a 
n  n-1  n- 


b  W(n“2)(0) 
n 

b  W^n_1^(0) 
n 


+ . . .+  b3  W(0)  +  b2  W(0) 
+ . . .+  b2  W(0)  +  bx  W(0) 


a 


1 


a 


o 


all  simultaneously,  then  the  integral  in  (1.7)  vanishes 
identically,  and  y(t)  is  then  a  particular  solution  to  (1.1). 
The  general  solution,  from  elementary  differential  equations 
can  then  be  expressed  as  the  sum  of  y(t)  and  the  general 
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solution  to  the  reduced  differential  equation,  the  latter 
approaching  zero  as  t — >co  .  Thus,  if  we  wait  long  enough 
after  the  initial  instant,  the  output  will  be  approximated 
as  closely  as  we  please  by  y(t).  This  is  what  is  meant  by 
the  s  eady  state  solution.  If  we  have  not  waited  very  long, 
the  difference  between  the  output  and  the  steady  state  solu¬ 
tion  is  called  a  transient . 

The  equations  (19)  can  be  solved  for  W(0) ,  ..., 
w(n-D(o)  uniquely  since  bn  /  0.  Thus  W(x)  is  a  solution 
to  our  problem  if  it  is  a  solution  of  the  reduced  differential 
equation  v ith  specified  initial  conditions.  That  such  a 
solution  exists  and  is  unique  follows  from  differential  equa¬ 
tion  theory.  We  have  now  found  a  function  W(x)  such  that 

CD 

(1.1J)  y(t)  J  W(x)  x  (t-T)  dx 

o 


is  the  steady  state  output  of  the  filter.  It  can  also  be 
shown  that  there  is  only  one  such  function  W(t)  with  this 
property.  (The  proof  depends  on  complex  variable  theory,  and 
is  omitted ) . 

Note  that  W(t)  will  decay  exponentially  to  zero  as 

'  — >  CD  . 


If  we  allow  the  numerator  of  F(S)  to  have  degree 
equal  to  or  larger  than  that  of  the  denominator,  the  previous 
analysis  breaks  down.  The  concept  of  weighting  function  can 
be  extended  even  so,  however,  but  not  without  introducing  the 
Dirac  delta  function.  The  W(x)'s  are  no  longer  nice  func¬ 
tions  in  the  usual  sense,  and  their  manipulations  are  fraught 
with  hazards.  Even  a  competent  person  in  filter  theory  must 
occasionally  exercise  caution,  although  he  is  generally  quite 
familiar  with  delta  functions  and  their  admissable  manipula- 
t ions  . 

Transfer  functions  whose  denominators  have  roots 
with  positive  real  parts  are  seldom  purposely  used  in  missile 
work  as  they  represent  unstable  filters.  However,  quite 
often  it  is  useful  to  consider  roots  with  real  parts  equal  to 
zero.  (For  example,  1  would  formally  represent  a  pure  in- 

tegrator.)  If  one  pushes  the  root  slightly  to  the  left  in  the 


r* 
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complex  plane  (say  g^~)  .  proceeds  with  the  analysis,  and  then 

finally  lets  e-^>0,  one  can  justify  many  of  the  manipulations 
executed  by  a  competent  servo  man,  who  generally  neglects 
this  necessary  logical  step.  He  knows  by  experience  what  he 
can  and  cannot  do  with  these  transfer  functions. 


One  can  also  justify  the  delta  function  manipula¬ 
tions  for  the  case  mentioned  previously  by  adding  a  few  small 
higher  order  terms  to  the  denominator  to  reduce  the  transfer 
function  to  the  case  we  have  considered,  and  then  at  the  end 
letting  these  terms  approach  zero. 


In  general,  we  shall  try  to  avoid  these  somewhat 
pathological  transfer  functions,  leaving  the  pursuit  of  them 
to  the  interested  reader. 


Proof  of  B 


write 


Applying  integration  by  parts  liberally,  we  can 


i.n)  j  bo 

o 


OO  „  (X) 

e  b  W(t )  dT  =  bQJ  e-bT  W(r)  d 


o 


QO 


CD 


/  bx  e“ST  W(t)  dx  =  b1  sj  e  St  W(t)  dx  -  bx  W(0) 


00 


00 


/ 

o 


bQ  e"ST  W(t)  dx  =  bQ  S 2  J  e  StW(t)  dT  -  b2  W(0) 


o 


-b2  S  W(0) 


f  bn  e"ST  W(n)(x)  dT  =  bn  S nJ  e  3Sv(t)  dT  -  bn 
J  o 


QD 


-St 


W(n_1)(0)  -  bn  SW(n-2)(0)  -...-  bnSn“1W(0) 
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Adding  all  of  the 

have  0  =  (b  +  b,  S 

o  1 

(1 • 12) 

-  (a  .  Sn_1 
n-1 

Solving  for  the  integral  yields  the  desired  result.  The 
above  operations  are  legitimate  as  long  as  the  real  part  of 
S  is  greater  than  the  real  part  of  every  root  of  the  denomi¬ 
nator  of  F(S) . 

Proof  of  C 


above,  using  (1.8)  and  (1.9),  we 

+  ...+  bn  S nj  e  ^TW(t)  dx 
o 

+  .  .  .  +  aj^  S  +  aQ)  . 


This  is  obvious,  and  the  proof  is  omitted. 

Proof  of  D 

Since 

(1.13)  A  cos  ot  =  ^  (ei<Jt^  +  e  i<Jt), 

we  have,  from  (1.10)  and  statements  B  and  C, 

ao 

(1.14)  y  ( t )  «|/  W(T)  [eiu(t'T)  +  e"lu(t"T)]  dx 


co 

A  iut  f  V,T(  x)  e  itJ  r  dx 
2  G  o 


oo 

x  A  e~iut[  W(x)  eiut  dT 
2  o 


■=  [e*Ut  F(iw)  +  e  F(-iw) ] 


^  [cos  ut  +  i  sin  ut  ]  [  IleF(  iu)  +  i  Im  F(iu)l 
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+ 


A 

2 


[cos  <Jt 


i  sin  ut]  (ReF(iu) 


i  Im  F( iu)  ] 


=  A  [Re  F(iu)  cos  ut  -  Im  F(iu)  sin  ut  ]  . 


The  result  follows  by  setting 


(1.15) 


cos  f6 


Re  F( iu) 
F(  iu) 


and 


(1.16) 


sin  d> 


Im  F( iu) 
F  ( iu ) 


Discussion - Statement  D  is  a  key  to  the  usefulness 

of  the  concept  of  a  transfer  function,  and  to  why  the  word 
"filter"  is  used.  It  tells  us  what  happens  to  the  output  of 
^  filter  in  steady  state  when  the  input  is  a  sine  wave. 

may  be  large  for  some  u's  and  small  for  others.  Since 


I F(iu) 

the  ratio  of  the  amplitude  of  output  to  input 
we  see  that  when 
that  frequency,  i 


X  A  W  V  A.  V  V  w  W  A  ••  M  V 

v(iu)|  is  small,  the  filter 


is  just 
"filters 


F(iu)| 
out" 


it  greatly  attenuates  its  amplitude. 


When  ore  uses  transfer  functions,  one  is  commonly 
said  to  be  "working  in  the  frequency  domain",  mainly  because 
of  the  above  paragraph,  i.e.,  because  of  the  natural  con¬ 
nection  between  behavior  of  filters  in  the  presence  of  sine 
wave  inputs  and  pure  imaginary  values  of  S.  When  one  analyzes 
filters  by  use  of  their  weighting  functions  alone,  one  is 
said  to  be  "working  in  the  time  domain".  The  conversion  from 
one  domain  to  the  other  is  seen  by  statement  B  to  be  accomp¬ 
lished  by  a  Laplace  (or  Fourier)  Transform.  (The  conversion 
from  frequency  to  time  depends  on  the  theory  of  inverse  trans¬ 
forms,  which  we  shall  not  touch  upon  specifically  in  this 
paper)  . 

Exercise  (a)  Consider  the  transfer  function  F(S)  =  _1 —  . 
-  - —  1+S 

Graph  F(iu)  and  arctan  versus  u. 
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Kxerc  iso  ( b)  Let  F^(F)  and  ^2(8)  be  two  stable  transfer  func¬ 
tions  whose  numerators  are  of  lower  degree  than 
the  denominators  Let  x(t)  be  the  input  to  the 
first,  y(t)  its  output.  Let  y(t)  be  the  input 
to  the  second,  z(t)  its  output.  Prove  that  there 
exists  a  filter  with  transfer  function  F„(S) 
whose  input  is  x(t)  and  wh^se  output  is  z(t), 
and  that  Fg(S)  =  F^(S).  Prove  also  that,  for  the 

steady  state  results  developed  in  this  section, 
it  is  legitimate  to  cancel  common  factors  of  the 
numerator  and  denominator  of  Fg(S). 

Exercise  (c )  If  either  F^(S)  or  FgtS)  are  unstable,  and  cancel¬ 
lation  of  factors  in  F^(S)  reduce  F3(S)  to  a 

formally  stable  transfer  function,  show  that  it 
is  generally  not  possible  to  carry  over  the  steady 
state  results  of  this  section  to  Fg(S). 


II.  TIME  SERIES,  STATIONARY  TIME  SERIES ,  ERGODIC  STATIONARY 
TIME  SERIES,  AUTOCORRELATION  FUNCTIONS,  POWER  SPECTRAL 
DENSITIES,  NOISE 


By  a  time  series  we  mean  an  ensemble  (or  collection, 
aggregate,  population,  class,  set,  etc.)  of  functions  xa(t), 

(a  is  the  index  which  varies  over  the  ensemble)  defined  for 
-00  <  t  <  03  ,  where  a  probability  distribution  exists  over  the 
a's.  Signals  about  which  we  have  only  statistical  knowledge 
(such  as  noise,  target  maneuvers)  are  treated  as  being  time 
series,  since  the  theory  of  time  series  seems  to  be  the  only 
known  mathematical  theory  whose  results  agree  with  observation, 
and  by  which  one  can  design  intelligent  filters  to  operate  with 
such  things  as  inputs.  A  noise  trace,  say  from  missile  tele¬ 
metering  signals  of  off-beam  error,  for  example,  defined  for 
tQ  <  t  <  t^,  is  then  regarded  as  a  section  of  a  particular 

sample  function  of  a  time  series. 

By  the  autocorrelation  function  of  a  time  series  we 
mean  the  function  Eatxa(t)  xa( t+ t)  J  ,  which  is  a  function  of 

t  and  t.  Thus ,  to  determine  the  value  of  the  function  for 
given  fixed  t  and  r,  we  compute  the  mathematical  expectation 
(or  mean,  or  average)  over  all  a  of  xQ(t)  x^it+r). 

If  a  time  series  is  s tat  ionary ,  we  think  of  the  time 
series  as  having  the  same  statistical  properties  if  all  func¬ 
tions  of  the  ensemble  are  shifted  to  the  right  or  left  by  the 
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same  amount  on  the  litre  axis.  For  a  stationary  time  series, 
the  autocorrelation  1' unction  is  a  function  of  t  alone,  and 
not  of  t.  This  simplication  simplifies  the  mathematical 
theory  tremendously,  and  fortunately,  many  statistical  sig¬ 
nals  found  in  missile  work  can  be  regarded  as  stationary. 
Unless  explicitly  stated  otherwise  in  the  sequel,  it  is  as¬ 
sumed  that  all  time  series  are  stationary. 


The  knowledge  of  the  autocorrelation  function  (or 
equivalent  functions  from  which  it  can  be  derived)  or  some 
approximation  to  it  is  required  to  do  intelligent  design 
work  in  many  instances.  With  no  a  priori  knowledge,  one  must 
resort  to  measurements.  It  is  not  hard  to  visualize  the 
problems  connected  with  trying  to  look  at  a  lot  of  different 
samples  simultaneously,  being  sure  they  are  samples  of  the 
same  time  series,  and  then  taking  ensemble  averages.  Fortu¬ 
nately,  there  is  a  convenient  crutch  called  ergodicity  which 
enables  one  to  avoid  this.  When  looking  at  a  sample  trace 
of  noise,  one  is  sorely  tempted  to  believe  that  one  can  aver¬ 
age  over  time  instead  of  over  space  (ensemble).  So  one  postu¬ 
lates  that  the  time  series  is  ergodic  (this  term  is  applied 
only  to  stationary  time  series)  and  writes 

T 

l  r 

lim  /  x  (t)  x  (t+r)  at  (for  any  fixed  a) 

£  I  J  CL  CL 

-T  =  E  [ x  ( t )  x  ( t *  r)  ]  ( f or  any  fixed  t). 

rr,  _  a  a  a 

r  — >  od 


The  autocorrelation  function  is  then  computed  by  the  left  hand 
side  using  a  single  noise  trace.  Suffice  it  to  say  that  ex¬ 
cept  in  cases  where  the  time  series  is  obviously  not  ergodic, 
one  rarely  gets  into  trouble  by  assuming  that  it  is.  Wiener, 
in  his  classic  work  on  stationary  time  series  (very  difficult 
to  read)  assumes  ergodicity,  but  for  most  of  his  work,  this 
was  an  unnecessary  assumption.  The  only  practical  reason 
for  assuming  it  is  when  one  wishes  to  measure  an  autocorrela¬ 
tion  function  (or  its  equivalent).  Since  we  will  not  be  con¬ 
cerned  with  measurement  in  the  sequel,  we  will  not  assume 
ergodicity.  Henceforth,  unless  stated  otherwise,  all  time 
series  are  assumed  to  be  stationary,  not  necessarily  ergodic, 
and  all  averages  are  ensemble  averages. 


Stationary  Time  Series - Let  x(t)  be  a  stationary 

time  series,  (the  subscript  a  omitted  for  brevity)  and  let 
A  (r)  denote  its  autocorrelation  function.  The  A  (r)  has  the 

X  X 

following  properties: 


i 
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E.  A  (0)  -  o’* ,  the  expected  value  of  the  square  of 

X 

x(t)  and  hence  Ax(0)  >  0.  (Why?) 


F.  A  (t)  -  A  ( — t )  for  all  t. 

X  X 

in  the  definition  of  Ax(t) 


G.  Ax(t) 


<  A  (0)  for  all  t. 
=  x 


by  the  Schwarz  Inequality, 
We  omit  the  proof. 


(Replace  t  by  t-r 

). 

This  can  be  proved 


The  power  spectral  density  (abbreviated  by  P.S.D. 
henceforth)  denoted  by  <zj!  (s) ,  is  defined  to  be 

A 


(2.1) 


0X<S) 


A  (t)  dx  .  (Laplace  transform  of 

A 


the  autocorrelation  function)  • 

For  a  wide  class  of  time  series, 

-oo 

verges,  and  when  this  happens,  0  (S)  exists  when  S  is  a  pure 

A 

imaginary  number,  which,  as  one  might  suspect,  is  the  case  of 
main  interest.  In  many  cases  0V(S)  is  a  rational  function, 

and  can  be  defined  over  the  whole  complex  plane  (not  in  general 
by  the  integral,  but  by  analytic  extension).  As  before,  we 
shall  assume  that  our  future  manipulations  are  legitimate,  and 
not  be  too  concerned  about  rigor. 


oo 


/ 


VT) 


dr  con- 


0  (S)  has  the  following  properties,  which  we  will 

A 

discuss  in  turn: 

H.  ^X(S)  is  real  for  real  and  pure  imaginary  S. 

I.  0X(S)  =  ^IsT. 
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J. 

^X(S)  - 

■  <*xes). 

,  r00 

K. 

—  f 

2* 

2 

(iw)  dm  -  o 

A 

L. 

If  the 

time  series  x(t)  is  passed  through  a 

stable  filter  with  transfer  function  F(S) ,  the  output  y(t)  in 
steady  state  is  a  time  series  whose  P.S.D.  is  given  by  dy(S)  ■ 

F(iw)|  2  rfx(iu). 


F(S)  F(-S)  f!<x(S). 


In  particular,  (rfy(iw) 


M.  dx(S)  >  0  for  pure  imaginary  S. 

Proof  of  H 

That  d  (S)  is  real  for  real  S  follows  directly  from 
X  -S  T 

(2.1).  If  S  =  iw,  e  =  cos  ut  -  i  sin  ut  .  Since  sin  u t 
is  an  odd  function,  it  follows  from  F  that  the  imaginary  part 
of  (2.1)  integrates  to  zero. 


Proof  of  I  ao 

Let  S  =  U  +  iV.  Then  4  (S)  =  J  e“UT(cos  Vx  -  i 

x  _ 

“00 

sin  Vx )  A  (x)  dT  . 


From  here,  I  is  obvious  . 

Proof  of  J 

By  making  a  change  of  variables,  and  using  F, 

00  “OD 


-/  eSTA, 


“OD 


(x)  dx  = 


A  ( -U)  dU 
2\ 


00 

eSU  Ax(-U)dU  =/  e"SUAx(U)  dU  =  dx(S). 
“00 


Proof  of  K 
oo 


0( iu)dw 


W  r  cd 


■ir-  lim  / 

"7T  v  _  in 


W->co  -W 


/ 

“00 


e  icJT  Ax(x)  dx 


dm 
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(2.2) 


(2.3) 


I_ 

2 17 


OD 

w 

lim  f  A  (x) 

f  fi-iur  dwx 

Yi->coJ 

-oo 

J 

-W 

00 

lim 
W  ->oo 


/ 


-CD 


VT) 


sin  Wx 
T 


dx 


-  lim 
17  W-»oo 


00 

/ 

-00 


A„  £ 


sin  U 
x  IW/  U 


dU  (letting  U  =  xW) 


oo 

-oo 


lim  £ 


U\  sin  U 


W->oo 


w  u 


dU 


oo 


oo 


i  /  *x<0)  ^  dU  -  Ax(0)  i  /  dU  = 

-oo  “00 


V°>  ■  °2 ' 

Proof  of  L 


In  steady  state,  we  can  write,  from  A, 
ao 

y(t)  =  J  W(U)  x(t-U) dU  and 
o 


oo 

y( t+x)  =  J  W(V)  x(t+x-V)  dV. 
o 

Multiplying  the  above  equations  together, 
oo  oo 


y(t)  y(t+x) 


W(U)  W(V)  x  (t-U)x  (t+x-V)  dU  dV. 


o  o 


Taking  expectations,  t  disappears  from  the  right 
hand  side  (as  will  be  seen  from  the  following  equalities)  and 
we  write: 
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(2.4) 


A  (  t) 

y 


(Integrate  average  rather  than 
oq  qd  average  integrals.) 

E  J  J  W(U)W(V)  x  (t-U)x  (t+T-V)dU  dV 


o  o 


OO  OD 


J  J  W(U)  W(V)  E[x(t-U)  x  (t+T-V)l  dU  dV 


o  o 


00  OO 


J  J  W(U)  W(V)  Ax(t+U-V)  dU  dV. 


O  o 


y(t)  is  also  stationary,  and  the  fact  that  t  disappeared  is 
in  agreement  with  this,  although  it  is  not  a  proof.  The  care¬ 
ful  reader  will  note  that  we  have  not  rigorously  defined 
stationarity ,  and  for  obvious  reasons  we  have  therefore 
omitted  the  proof  that  y(t)  is  stationary. 

Finally, 


w 

(2.5)  ty( S)  =  J  e"ST  Ay(T)  dx 


00  f~00  OD 

r  -si  r  r 


=  /  e_ST/  f  W(U)  W(V)  Ax(x+U-V)  dU  dV  dx 


o  o 


ao  oo 


■// 

o  o 


-S(V-U) 


W(U)  W(V)  e 


Ax(x+U-V)  dx  dU  dV 


oo  oo 


-S(x+U-V) 


=  f  f  W(U)  W(V)  e"SV  eSU  [|rfx(S)  ]  dU  dV 


o  o 


-«<X(S)  / 


W(U)  eSU  d 


F (S )  F(-S)  6  (S)  . 

A 


W(V)  e'SV  dV 


(Laplace  transform  of 
weighting  function.) 
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The  result  for  S  -  i  follows  from  C 
Proof  of  M  (Intuitive) 


If  ^x(iu0)  were  negative,  we  choose  a  stable  transfer 
If ( itj )|  ^  is  minutely  small  everywhere 


function  F(S)  such  that 


except  near  uQ  (and  -uQ).  Then 


siL  iF(iw) 


0(  tu)  d u  <  0. 


If  we  send  x(t)  through  the  filter,  the  expected  value  of  the 
square  of  the  output,  is ,  by  K  and  L,  just  the  above  integral 
On  the  other  hand,  the  expected  value  of  a  square  is  never  nega¬ 
tive,  a  contradiction. 


Discussion - Statements  K  and  L  are  a  key  to  the 

utility  of  the  concept  of  P.S.D.  If  we  know  the  P.S.D.  of  an 
input  to  a  filter  (which  can  be  computed  from  the  autocorrela¬ 
tion  function  by  (2.1))  we  know  the  P.S.D.  and  mean  square  of 
the  output.  Since  missile  motion  can  be  considered  as  the  out¬ 
put  of  a  filter,  one  sees  that  we  have  a  powerful  tool  for  com¬ 
puting  mean  square  miss  distances  when  the  P.S.D.  or  auto¬ 
correlation  function  of  the  input  is  known.  Accuracy  of  track¬ 
ing  radars  as  well  as  the  performance  of  other  devices  can  be 
analyzed  using  these  concepts. 

In  many  practical  cases,  the  integrals  can  be 
evaluated  in  closed  form  using  residue  theory,  and  tables  exist 
from  which  they  can  be  quickly  computed. 

Two  examples  of  time  series  are  target  acceleration 
(say  in  one  coordinate)  and  noise.  Clearly  target  acceleration 
must  generally  be  regarded  as  statistical,  since  the  pilot  is 
not  going  to  tell  us  what  he  is  going  to  do.  Noise  comes  from 
many  sources,  and  is  always  present  to  some  degree  in  trans¬ 
mission  and  reception  of  radar  energy.  There  is  glint  noise, 
fading  noise,  receiver  noise,  and  others. 

There  are  two  types  of  P.S.D. 's  which  have  been  given 
names,  and  are  worth  mentioning.  The  first  is  &(S)  =  16 ,  a  con¬ 
stant,  and  the  time  series  is  said  to  be  "white."  One  some¬ 
times  uses  the  phrase  "white  P.S.D."  or  "white  noise."  Such 
a  time  series  cannot  exist  in  nature,  because  a 2  is  infinite 
(from  K).  It  is  nonetheless  a  useful  artifice.  If  0(S)  takes 

2 

the  form  0(S)  =  — ^ - g-,  a,  b  real  and  different  from  zero, 

b  -  S 

one  substitutes  the  work  "Markovian"  for  "white."  Note  that 
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d(S)  is  the  P.S.D.  of  the  output  of  the  filter  F(S)  “ 

when  the  input  has  P.S.D.  "  1.  Although  "white  noise' 
exist  in  nature,  "Markovian  noise"  can. 

The  words  "power  spectral  density"  are  well  chosen. 

If  a  filter  F(S)  "filters  out"  all  frequencies  except  those  in 
a  small  interval,  K  and  L  show  that  the  output  power  (°y  ) 

is  proportional  to  d(i u)  for  u  in  the  interval  and  proportional 
to  the  "bandwidth,"  or  the  length  of  the  interval.  Thus,  P.S.D. 
is  power  per  unit  bandwidth.  It  can  be  seen  that  a  white  time 
series  has  the  same  power  per  unit  bandwidth  for  all  fiequencies. 
A  Markovian  time  series  has  a  P.S.D.  which  looks  like  a  white 
P.S.D.  for  small  frequencies  and  decays  away  for  large  fre¬ 
quencies  . 

2 

Exercise  (d)  If  d(S)  =  j  ,  a,  b  real  and  different  from 

2  b  -  S 

zero,  find  a. 


III.  OPTIMUM  FILTERS  AND  THE  WIENER  HOPF  EQUATION 


I  a  1 

|  b  |  +  S 
'  cannot 


Suppose  the  input  to  a  filter  is  composed  of  the  sum 
of  two  time  series,  x(t)  +  N(t),  where  x(t)  is  regarded  as  the 
"signal,"  or  desirable  part  of  the  input,  and  N(t)  is  the 
"noise,"  or  undesirable  part  of  the  input.  Suppose  that  the 
P.S.D. 's  or  the  autocorrelation  functions  of  x(t)  and  N(t)  are 
not  known,  and  that  x(t)  and  N(t)  are  statistically  independent 
and  have  mean  zero.  This  implies  that 


(3.1)  E  [x(t^)  N  ( t  g )  ]  =  0  for  all  and  tg  ■ 


We  now  wish  to  design  the  filter  (with  weighting 
function  W(t)  and  transfer  function  F(S))  which  makes  the  out¬ 
put  y(t)  resemble  the  "signal"  portion  of  the  input,  x(t), 
as  closely  as  possible.  The  criterion  used  by  Wiener  is  that 
the  steady  state  value  of  E  (y(t)  -  x(t))^  should  be  a  mini¬ 
mum.  As  we  shall  see,  this  leads  us  to  a  problem  in  calculus 
of  variations.  Let  us  denote  the  above  expression  by  o  . 

Then,  since 


(3.2)  y (t)  =  J  W(t) [  x(t  -  t)  +  N (t  -  t) ]  dx  - 

o 
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we  have,  by  subtracting  x(t)  from  both  Jides  and  squaring, 

oo  oo 

(3.3)  [  y  (t  )  -  x  (t  )  ]2  -  J  J  W(t)  W(U)  [x  (t  -  t) 

o  o 

+  N (t  -  t)]  [x (t  -  U)  +  N (t  -  U)]dT  dU 
CD 

-  2  J  x (t )  W(t)  [  x(t  -  t)  dr 
o 

+  x(t) ]2  . 


Let  us  denote  the  autocorrelation  functions  of  x(t) 
and  N (t )  by  ax(t)  and  V  x)  respectively,  and  the  P.S.D.'s  of 

x(t)  and  N (t)  by  jz$x(S)  and  iz$N(S),  respectively.  Averaging 

both  sides  of  (3.3),  we  have 


oo  oo 


(3.4)  o2  =  J  J  W(t)  W (U)  [  Ax(x  -  U)  +  An(t  -  U)  ]  dr 


dU 


o  o 


00 


-2  J  W(t)  Ax(t)  dx  +  Ax(0)  . 
o 

Exercise  (e)  Prove  ^hat  (Hint  :  Similar  to  Proof  of  K.) 
°2  =  J  |l  -  F(iu>)  2  ^x(iw)  du 


-CD 


oo 


+  Z?  / 


F(iw) 


j^N(iu)  dw. 


-oo 


Our  problem  mathematically  is  to  choose  W(t)  or  F(S) 
so  that  (3.4)  or  the  expression  in  exercise  (e)  is  minimum. 

The  classical  approach  is  to  work  with  (3.4),  although  the 
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problem  can  be  solved  by  working  with  the  expression  in 
exercise  (e)  directly.  The  latter  approach  is  somewhat  tricky, 

2 

however,  as  care  must  be  taken  to  optimize  o  over  stable 
transfer  functions.  This  is  automatically  taken  ;are  of  in 
the  first  approach  by  virtue  of  the  definition  of  weighting 
functions.  We  shall  here  use  the  first  approach. 

Suppose  that  W(t)  is  such  that  (3.4)  is  a  minimum. 

If  rj  (t)  is  an  arbitrary  fixed  well  behaved  function  of  time, 
and e  an  arbitrary  number,  we  cannot  then  get  a  smaller  value 

for  o2  if  we  replace  W(t)  by  W(t)  +  ty (x)  in  (3.4). 

That  is,  if  we  write 


00  00 

(3.5)  a2  ( c )  -  J  J  [W(t)  +  €  tj  ( x )  ]  [W(U)+  eT](U)][Ax(r  -  U) 

o  o 

+  An ( t  -  U)]  dx  dU 
00 

-2  J  [W(t)  +  ctj(t)]  Ax(t)  dx  +  Ax  (0). 
o 

2 

Then  a  (e)  must  be  a  minimum  for v6  =  0.  So  we  differentiate 

(3.5)  with  respect  to  ,e  ,  and  set  €  =  0,  obtaining 

oo  a> 

(3.6)  °  =  J  J  W(x)  tj(u)  [Ax(x  -  U)  +  An(t  -  U)]  dx  dU 

o  o 

00  oo 

+  J  J  W(U)  T(t)  [Ax(x  -  U)  +  Ajj(x  -  U)  ]  dx  dU 
o  o 

00 

-2  f  7](x)  A  (x)  dx  . 

J  x 

o 
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Now  the  second  integral  is  the  same  as  the  first, 
since  Ax(x  -  U)  +  A^(x  -  U)  =  Ax(U  -  x)  +  A^(U  -  t) ,  so  we 

put  a  2  in  front  of  the  second  integral  and  throw  the  first 
one  away.  Dividing  by  2,  we  have 

oo  f-  oo 

(3.7)  J  J  j  W(U)  [Ax(t  -  U)  +  AjjCt  -  U)]  dU 
o  o 


-  Ax(T)j  t|(t)  dx  =  0  . 

The  expression  within  the  braces  must  vanish  identi¬ 
cally  for  all  positive  values  of  x,  for  if  it  did  not  vanish 
for  x  -  xq  say,  one  could  choose  a  function  tj(x)  which  was 

positive  in  the  neighborhood  of  x  =  xq  and  zero  elsewhere, 

and  (3.7)  would  not  then  vanish.  Since  (3.7)  must  hold  for 
arbitrary  n(x),  we  conclude  that 

oo 

(3.8)  J  W(U)  [Ax(x  -  U)  +  AN(x  -  U)]  dU  =  Ax(x)  for  x  >  0. 
o 

This  is  one  form  of  the  Wiener-Hopf  Equation.  Its 
significance  is  that  the  optimum  weighting  function  must  obey 
it . 


Solving  for  W(x)  or  F(S)  explicitly  is  somewhat  tricky. 
We  proceed  as  follows:  Let  W(U)  be  0  for  negative  U,  and  the 
lower  limit  O  in  the  integral  of  (3.8)  can  then  be  replaced 
by  -oo  . 


Since  (3.8)  does  not  tell  us  what  the  integral  is 
equal  to  for  negative  x,  we  write 

oo 

(3.9)  J  W(U)  [Ax(x  -  U)]+  Ajj(x  -  U)]  dU  =  Ax(x)  +  r(x) 

-oo 

where  r(x)  vanishes  for  positive  x,  and  is  otherwise  unknown. 
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Next,  we  multiply  (3.9)  by  e'ST  and  integrate  from 


-CD  tO  00  . 


If  we  set 


(3 


.10)  R(S)  «  J  e"ST  r (t) 


dx,  we  have 


-CD 


oo  r  00 

(3.11)  e)  (S)  +  R(S)  -  f  e  St  <  / 

*  (-00  W(U)  [Ax(t  -  U) 


-OO 


+  A^x  -  U)]  dU ^  dr 

oo  r  00  x 

/  «(D)  e'SU  j  /  e‘S(T  -  U)[Ax(t  -  U) 


-CD 


+  Ajj(t  -  U)]  dx 


■oo 


y  dU 


co  r  00 

/  W(„)  e-SU  / 

■oo  L_gd 


e'SV  [AX(V) 


*N<V)1 


U  ' 


dV  >  dU 


(U)  e'SU  dU 


oo 


f  e*SVlAx(V) 


-CD 


VV>1 


dV 


=  F(S)  [  si  (S)  +  ^N(S)]  • 
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So  the  optimum  transfer  function  satisfies 


(3.12)  F(S)  [6  (S)  +  dw(S)]  -  6  (S)  +  R(S)  . 

X  IN  X 


At  first  sight  this  looks  like  a  useless  expression 
since  R(S)  is  unknown.  It  is  not  completely  unknown  however. 
Let  us  assume  that  r(r)  decays  to  zero  as  t  — > ao  sufficiently 
rapidly  that  the  integral  in  (3.10)  exists  for  S  *  i  .  From 
(3.9),  this  is  reasonable  if  A  and  Aj.  are  sufficiently  well 
behaved.  Inspection  of  (3.10)xthen  snows  that  R(S)  exists 
whenever  the  real  part  of  S  is  negative.  In  practice,  R(S) 
turns  out  to  be  a  rational  function.  If  it  becomes  infinite 
at  all,  it  can  only  become  infinite  when  the  real  part  of  S 
is  positive.  (Note  that  this  is  just  the  reverse  for  F(S).) 

It  turns  out  that  this  property  of  R(S)  is  sufficient 
to  determine  F(S)  uniquely,  using  (3.12).  The  actual  explicit 
expression  for  F(S)  would  take  us  far  afield  into  notations 
for  factors  of  dx(S)  +  ^(S),  removal  of  singular  parts,  and 

Liouville's  theorem  in  analytic  function  theory.  We  shall  not 
attempt  to  derive  the  general  explicit  form  for  F(S)  here. 

However,  it  will  be  seen  in  the  next  section  how,  in 
a  given  case,  F(S)  can  be  found. 


IV.  AN  OPTIMIZATION  EXAMPLE:  DESIGN  OF  A  TRACKING  RADAR  ANGLE 
LOOP 


Suppose  an  aircraft  is  at  an  elevation  angle  0^,. 

When  we  attempt  to  measure  9^,,  however,  we  measure  instead  0^, 

+  0„ ,  where  0XT  is  noise.  We  now  wish  the  radar  direction, 

N  N 
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©  ,  to  be  as  close  to  ©_  as  possible.  Using  the  least  squares 
R  T 

criterion,  the  results  of  the  last  section  can  be  seen  to  be 
applicable . 

..  We  shall  assume  that  ©N  has  a  white  P.S.D.  equal  to 
(6,  and  ©T  has  a  white  P.S.D.  equal  to  ©.  These  are  sometimes 

reasonable  assumptions. 

It  is  possible  to  get  into  trouble  with  these  assump¬ 
tions  because  ©T  itself  has  a  P.S.D.  which  is  infinite  for 

S  =  0.  Also  Ajj(t)  is  infinite  for  t  -  0.  This  means  that 

some  of  the  manipulations  of  the  previous  section  which  all 
assumed  well-behaved  functions,  are  not  strictly  legitimate. 


We  get  around  this  by  saying  that  <z$N  has  a  P.S.D. 


rr—rr  (instead  of  tf)  and  ©_  has  a  P.S.D. 
^  1 


0 


(S~  -  6) 


j(instead 


1  -  e  S‘ 

of  -j)  and  eventually  letting  €  and  <5  zero  (assume  e  >0, 

S 

6  >  0)  .. 

From  (3.12)  then, 

,,  wox  r  ©(1  -  eV)  +  rf(S2  -  d2)21_  ©  . 

(41)  F<S>  L  a  -  « V)  (s2  -  aV  j  (S2  -  7?  m 


If  we  imagine  both  sides  of  (4.1)  expressed  as  the 
sum  of  partial  fractions,  we  see  that  R(S)  must  take  the  form 


(4.2) 


R(s>  -  snhr + 


B  C 

i  ^  j  g  g  +  polynomial 


(S  -  6) 


Let  €  ,  o->0,  we  have 


(4.3)  F(S)  —  +^S  =  ^  +  polynomial 

S  S  s 
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or 

(4.4) 


F(S)  = 


8  +  BS2  +  AS3  +  S4  (polynomial) 
8  +  ^S4 


From  exercise  (e)  we  see  that  the  polynomial  must  be 
zero,  for  otherwise  the  second  integral  there  would  be  infinite 


(4.5) 


So,  since  the  denominator  factors, 
F(S)  = 


8  +  BS2  +  AS3 


(Ve  +  \/2  ey40V4  s  +  Vrf  s2)  (Ve-  /2  ©J/4  V4  +S  s2) 


Since  F(S)  can  become  infinite  only  for  values  of  S 
which  have  negative  real  parts,  the  second  factor  of  the  de¬ 
nominator  must  be  an  exact  divisor  of  the  numerator.  If  one 
attempts  this  division,  one  will  discover  that  this  can  only 
happen  if 

(4.6)  e  +  bs2  +  as3  =  (Ve  +  /2  el4^1/4s)  (Ve  -  V2  e1/V/4s 
+  V*  s2) 


(4.7) 


Hence 


F(S) 


Ve  +  V2  6^% 1/4  s 
0  +  V2  e14/4  s  +  v^s2 


This  is  commonly  written  in  the  form 


(4.8)  F(S)  = 


1  +  2%tS 


1  +  2£xS  +  t2  S2 


where 


(4.9) 


t  =  1  , 

V2 


T  = 


0 


1 /4 
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From  exercise  (e)  and  the  use  of  tables,  the  minimum 
value  of  o^  can  be  computed  to  be 

(4.10)  o2  =  y[2  f^4 Q1/ 4  . 


The  problem  is  not  completely  finished,  however,  as 

the  radar  does  not  actually  measure  0T  +  0W,  but  0_,  +  0  0 

1  1  N  -  R 

By  the  action  of  the  electromagnetic  propagation,  the  measure¬ 
ment  is  in  the  form  of  boresight  angular  error.  This  measured 
error  must  then  be  used  to  physically  drive  the  radar  as  a 
servomechanism  until  the  error  is  nulled. 

The  operation  of  the  radar  might  then  be  as  indicated 
in  the  following  diagram 


0T+eN  _  °T+eN"0R 


Integrate 


measured  error 


knultiply  by 
2J?t 


multiply  by} 

-rz 


Drive  radar  at 
rate  equal  to 
this  signal 


action  of 
'adar  drivef 


(4 


Hence 


U)  ®R  '  ^2  12  T  <8T  +  «N  ‘  V  *  /  <8T  +  ®N  ’  V  dtl 


or 


(4.12)  t  0R  +  2  t  0R  +  0R  =  2  x  (0T  +  0N)  +  (0T  +  0N)  . 
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Hence,  we  have  the  equivalent  diagram: 


1M  O M  MONMI  IK«PT 

Am»  physics  laaocatocy 

HW  MMO  MAMLAMD 


eT  +  eN 


FILTER 


*R 


where  the  transfer  function  of  the  filter  is  given  by  (4.7)  or 
(4.8).  A  tracking  radar  behaving  as  just  described  will,  in 
the  least  squares  sense,  then  track  the  target  as  closely  as 
possible,  whenever  the  statistical  assumptions  are  justified. 
Other  statistical  assumptions  will,  of  course,  lead  to  different 
answers . 

Exercise  (f)  If  x(t)  is  a  stationary  time  series,  show  that 

00 

A  (t)  =7 r-  f  eiUT  fzT(itj)  d u.  (Hint:  see  proof  of  K.) 

X  Z1T  J  X 

-GO 


Exercise  (g)  In  the  example  of  section  IV,  assume  instead  that 
©T  is  white.  Find  the  optimum  transfer  function,  the  optimum 

mean  square  tracking  error,  and  draw  the  radar  block  diagram. 


Exercise  (h)  What  is  the  weighting  function  of  a  filter  whose 

transfer  function  is  -A—  ? 

1+TS 


Exercise  (i)  Let  x(t)  be  a  time  series  such  that 
Show  that  x(t)  is  Markovian,  ^se  this  result  and 

to  evaluate  the  integral  f  — cos  ut  d<j 

217  J  1  +  T 2  (j2 


Ax(t)  = 

exercise 


(f) 


SUMMARY  AND  DISCUSSION 


We  have  here  attempted  to  introduce  the  concepts  of 
filter,  transfer  function,  weighting  function,  time  series, 
stationarity ,  ergodicity,  autocorrelation  function,  power  spec¬ 
tral  density  and  noise.  We  have  also  introduced  the  Wiener 
theory  of  optimization,  derived  the  Wiener-Hopf  equation,  and 
have  shown  how  it  can  be  solved.  A  particular  application  has 
been  discissed  in  detail,  and  a  tracking  radar  angle  loop  has 
been  optimized  in  the  presence  of  an  input  which  has  been  cor¬ 
rupted  with  noise. 
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The  Wiener  theory  can  be  applied  to  other  cases,  al¬ 
though  the  derivations  in  this  paper  must  be  somewhat  altered 
to  achieve  greater  generality.  An  example  is  the  design  of 
an  optimum  loop  when  some  parts  of  the  loop  have  already  speci¬ 
fied  transfer  functions.  This  could  occur  in  the  design  of  a 
beam  riding  guidance  computer  where  the  aerodynamic  equations 
have  already  been  determined  by  the  airframe  design.  Wiener 
theory  can  also  be  applied  (by  the  use  of  Lagrange  multipliers) 
to  problems  with  constraints.  A  typical  problem  of  this  type 
is  the  design  of  a  beam  riding  guidance  computer  to  minimize 
deviation  from  the  line  of  sight  subject  to  the  constraint 
that  the  mean  square  acceleration  is  not  too  large.  (Or  else 
the  missile  might  slow  down  too  much  or  fall  apart.) 

In  general,  Wiener  theory  is  applicable  only  to  steady 
state  problems  where  the  time  series  may  be  considered  as 
stationary.  Because  of  the  mathematical  simplifications  which 
occur  for  this  case,  the  theory  is  quite  well  developed.  Al¬ 
though  a  considerable  amount  of  work  has  been  done  on  practical 
cases  for  which  Wiener  theory  does  not  apply,  the  general  case 
is  nowhere  near  as  well  understood.  For  example,  transient 
statistical  problems,  non-stationary  problems,  final  value 
problems  (  in  which  one  is  interested  in  the  output  only  at  a 
specified  time)  all  occur  in  missile  work,  and  seem  to  require 
special  handling. 

Wiener  theory  is  directly  applicable  to  a  large  frac¬ 
tion  of  statistical  design  problems  in  missile  work,  nonethe¬ 
less.  Some  people  prefer  to  take  a  different  approach  to  some 
of  these  problems.  One  such  approach  is  the  game  theory  ap¬ 
proach,  where  it  is  assumed  that  the  target  maneuvers  in  such 
a  way  as  to  maximize  the  tracking  error  (or  miss  distance), 
and  the  filter  is  designed  to  minimize  this  maximum. 

Another  different  class  of  problems  occurs  when  the 
time  series  under  consideration  are  defined  discretely,  i.e., 
say  at  multiples  of  At.  This  type  of  problem  occurs  with 
search  radars,  which  only  "look"  at  a  target  every  so  often. 

One  is  then  concerned  with  maintaining  track  on  this  target 
and  minimizing  the  tracking  error.  A  system  which  tracks  in 
this  manner  is  referred  to  by  some  as  a  "track  while  scan" 
system. 


Finally,  it  should  be  stated  that  although  the  results 
of  section  I  are  prerequisites  to  the  subsequent  sections,  they 
are  extremely  important  in  their  own  right.  Because  the  em¬ 
phasis  of  this  paper  has  been  on  time  series  and  optimization 
in  the  presence  of  noise,  it  might  appear  that  section  I  was 
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just  the  means  to  an  end.  Such  Is  not  the  case, 
and  properties  of  a  transfer  function  are  to  most 
gineers  as  the  ABC's  are  to  a  school  child.  They 
most  every  working  day  in  a  practical  sense,  even 
of  them  may  know  nothing  of  time  series  or  Wiener 


The  concept 
missile  en- 
use  them  al¬ 
though  some 
theory . 


It  is  safe  to  say  that  the  science  of  guided  missiles 
would  be  far  behind  its  present  state  of  development  if  it 
were  not  for  the  concept  of  a  transfer  function.  It  is  there¬ 
fore  strongly  suggested  that  the  reader  absorb  section  I  thor¬ 
oughly,  whether  or  not  he  has  the  time  or  patience  to  absorb 
the  rest  of  the  paper.  Section  II  is  of  secondary  importance, 
sections  III  and  IV  of  less  importance  still. 
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I.  A  WORD  ON  TAYLOR'S  SERIES 


If  y  -  f(x)  is  a 
on  an  interval  a  <  x  < 


twice  differentiable  function  defined 
b  containing  a  point  xq,  then 


(1)  y  -  f(x)  -  f(xQ)  +  f'(xo)  (x~x0)  +  \  fM  (V  (x-xq)2  ’ 


where  I-  is  a  point  between  x  and  x  .  This  is  a  form  of 
Taylor's  series  with  remainder.  The  remainder  term,  namely 


(2) 


1  ft  O 

2  f  (O  (x-xo)^  , 


represents  the  error  committed  when  the  first  two  terms  alone 
are ,used  as  an  approximation  to  f(x). 

V 


(3)  Error  -  |f(x)  -  f(xQ)  -  f  •  (xq)  (x-xq) 


(4) l(x-x0)2- 


The  number  (•  usually  is  itself  a  complicated  function  of 

x.  Normally,  the  only  thing  which  can  be  said  with  assurance 

about  £  is  that  it  lies  between  x  and  x  ,  and  hence  lies 
s  -  o 

in  the  interval  [a,  b] .  If  we  let  M  be  the  maximum  of 

I  f "  (x) |  for  a  <  x  <  b,  then  |f"  (£)  |  <  M  and  we  may  write 


(4)  Error  -  |f(x)  -  f(xQ)  -  f'(xQ)  (  x-xq) |  <  “  (x“x0>2. 


Thus,  although  it  is  usually  very  difficult  to  find  the  exact 
error,  it  is  fairly  easy  to  establish  a  bound  for  the  error. 
Usually  this  is  quite  sufficient. 
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The  form  of  Eq.  (4)  shows  that  the  error  will  be  small 
if  x  is  "close  enough"  to  x  .  Just  how  close  is  "close 
enough"  depends  on  how  much  e?ror  is  permissible  and  on  the 
magnitude  of  M. 


II.  LINEAR  APPROXIMATIONS 


(5)  f (x)  *  f(xQ)  +  f ’(xQ)  (x-xq) 


is  called  the  "linear  approximation  to  f(x)  at  x  -  xq"  inasmuch 

as  the  right  hand  side  is  linear.  Geometrically,  the  approxi¬ 
mating  function  is  the  tangent  to  y  -  f(x)  at  x  -  x^.  From 
the  preceding  discussion^  it.  is  evident  that  there  is  a  range 
of  values  of  x  "close"  to  x  for  which  the  linear  approxima¬ 
tion  is  adequate,  i.e.,  the  e?ror  committed  by  using  the 
approximation  lies  within  acceptable  bounds.  The  length  of 
the  range  depends  on  the  behavior  of  f(x)  in  the  neighborhood 
of  x  -  x  .  More  specifically,  if  f"  (x)  is  large  near  x  -  xq 
(which  means  that  the  curvature  is  large) ,  then  M  will  be 
large  and  the  length  of  the  range  will  small  (for  a  given  per¬ 
missible  error) .  On  the  other  hand  if  f "  (x)  is  small  near 
x  «=  xQ  (small  curvature)  ,  then  the  range  will  be  correspondingly 
large . 
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For  functions  of  more  than  one  variable  the  results  are 
quite  similar.  For  example,  the  linear  approximation  to 
w  -  w(x,  y,  z)  at  (x,  y,  z)  -  (xq,  yQ,  zQ)  is 


(6)  w(x,y,z)  a  w(xo.yo>*0)  +  A(x-xq)  +  B(y-yQ)  +  C(z-zQ) 


where 


(7)  A  -  Si 


3w  ^ 

3  X 


(8)  B  - 


—  > 
3x  f 


(9)  C  - 


3w 

3X  J 


evaluated  at  x-x  ,  y-y  ,  z«z^  . 

o  o  o 


As  before,  there  is  a  set  of  points  (x,  y,  z)  around  (xq,  yQ,  zq) 

for  which  the  error  in  the  approximation  is  small  enough  to  be 
acceptable.  Once  again,  the  actual  extent  of  the  point  set  de¬ 
pends  on  the  values  of  the  second  partial  derivatives  in  the 
vicinity  of  (xq,  yQ,  zq) . 

In  the  case  of  a  function  of  two  variables,  the  geometri¬ 
cal  interpretation  of  the  linear  approximation  is  readily 
available.  The  approximating  function  is  merely  the  tangent 
plane  to  the  surface  w  -  f(x,y)  at  (xq,  yQ) . 

III.  FORMULATION  OF  PROBLEMS 


The  mathematical  form  of  an  engineering  problem  is  an 
equation  or  a  set  of  equations  whose  solution  gives  the  desired 
answers.  Consider,  for  example,  the  following  simple  problem 
which  could  arise  in  connection  with  surveying.  Find  the 
angles  of  a  triangle  whose  three  sides  are  known. 
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The  mathematical  form  of  this  particular  problem  can  be  written 
as  the  following  three  equations. 


2  2  2 

(10)  a  -  b  +  c  -  2bc  cos  a  (law  of  cosines)  t 


(11)  s_ln_jj  .  sinjj  (u,  of  sines)  , 


(12)  a  +  /3  +  y  -  it 


It  should  be  noted  that  the  mathematical  form  of  the  problem 
is  not  unique.  Equation  (11),  for  instance,  could  be  replaced 

by 


2  2  2 

(13)  b  -  a  +  c  -  2ac  cos  (law  of  cosines). 


IV.  EQUATIONS  CONTAINING  PARAMETERS 


The  solution  of  an  equation  or  a  set  of  equations  usually 
will  depend  on  one  or  more  parameters.  In  the  surveying  problem 
just  considered,  the  solution  depends  on  three  parameters,  namely 
the  sides  of  the  traingle  a,  b,  c.  Other  examples  of  this  are 
given  in  the  following  equations. 
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(14) 

(15) 


3 


-  ax  + 


2  2 
+  y 


x  +  by 


6  - 

-  25 

-  5 


0 


(16) 

(17) 


ry  +  (3  +  ay  )  y  +  2y  -  0 
ly-1,  y--l  when  t  -  0 


1  parameter  -  a  , 


2  parameters  —  a,  b, 


1  parameter  -  a  , 


1  parameter  -  a  t 


(18)  rx y  -  1 


yx  -  ax  -  1 


t  -  o,  x  -  1,  y  -  1 


1  parameter  -  a  . 


The  first  three  examples  (Eqs .  14,  15,  and  16)  are 
ordinary  equations  for  which  the  solutions  are  numbers.  The 
last  two  examples  (Eqs.  17  and  18)  are  differential  equations 
for  which  the  solutions  are  functions  satisfying  certain 
initial  conditions. 


V.  METHOD  OF  LINEAR  PERTURBATIONS 


The  following  situation  very  frequently  occurs.  The 
solution  to  an  equation  is  known  for  certain  values  of  the 
parameters  appearing  in  the  equation.  It  is  desired  to  find 
the  solution  when  the  parameters  have  values  differing 
slightly  from  the  values  for  which  the  solution  is  known. 
There  is  a  fairly  standard  terminology  to  describe  this  state 
of  affairs.  "Knowing  the  solution  for  given  values  of  the 
parameters,  find  the  behavior  of  the  solution  when  the 
parameters  are  perturbed . "  The  word  perturbed  carries  the 
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connotation  of  differing  slightly . 

One  method  for  handling  the  problem  just  described  is 
the  method  of  linear  perturbations,  for  which  the  groundwork 
has  been  laid  in  the  preceding  paragraphs.  The  idea  is  quite 
simple.  The  solution  to  the  equation  is  a  function  of  the 
parameters  appearing  in  the  equation.  The  value  of  this 
function  is  known  for  certain  given  values  of  the  parameters. 
The  method  of  linear  perturbations  consists  of  replacing  the 
function  by  its  linear  approximation  at  the  known  point. 
Consider  the  following  example,  which  illustrates  the  method. 


VI ,  EXAMPLES 


Let  the  equation  to  be  solved  be 


(19)  x2  +  a  -  8x  . 


This  equation  has  one  parameter,  a.  When  a  -  12,  the  equation 
has  the  solutions  x  -  2  and  x  -  6.  What  will  be  the  solutions 
for  values  of  a  close  to  a  -  12?  The  solutions  are  functions 
of  a.  Indeed,  since  the  equation  is  a  quadratic,  we  can 
easTly  find 


(20) 


x  -  4  ±  Vl6  -  a. 


We  shall  consider  only  one  of  these  solutions,  namely 


(21) 


x  =  x(a) 


For  this  case  we  have 


(22) 


x(12)  -  6. 
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Furthermore , 


(23) 


dx 

da 


-  ,  so  that 

2~V  16  -  a 


(24)  /dx\  1 

W..12  '  *  ' 


The  linear  approximation  to  x(a)  at  a  -  12  is 

(25)  x(a)  -  x(12)  +  (£|)  (a-12)  -  6  -  ^  _  9  _  J  . 

a -12 

Equation  (25)  now  can  be  used  to  estimate  the  solution  for 
values  of  a  near  a-12.  Thus,  when  a  -  12.04,  we  obtain 
x  -  5.99.  The  exact  result,  of  course,  can  be  obtained 
from  Eq.  (21)  which  gives  x  -  4  +1 3 .96  -  5.989975  (making 
use  of  a  table  of  square  roots) .  We  now  see  that  the  error 
in  the  linear  approximation  is  about  5.99  -  5.989975  -  0.000025 

The  preceding  example  is  so  simple  that  it  is  apt  to  give 
rise  to  certain  misconceptions  which  it  would  be  well  to  dispel 
immediately.  Simple  as  Eq.  (21)  is,  Eq .  (25)  is  even  more 
simple,  so  that  there  is  a  clear  advantage  in  using  Eq.  (25) 
rather  Eq  (21).  Nevertheless,  Eq.  (21)  is  simple  enough  so 
that  there  can  be  no  real  objection  to  its  use.  Besides, 
apparently  we  need  Eq.  (21)  anyway  in  order  to  get  the  deriva¬ 
tive  which  we  used  in  Eq.  (25).  As  a  matter  of  fact,  this 
last  statement  is  false.  We  do  not  need  an  explicit  representa 
tive  of  x(a),  as  in  Eq.  (21),  in  order  to  find  the  derivative 
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^  .  This  is  very  fortunate,  since  usually  it  is  impossible 

to  find  such  a  representation.  In  such  a  case  we  use  implicit 
differentiation  to  find  the  required  derivatives. 

This  technique  would  be  applied  to  our  problem  in  the 
following  manner.  Equation  (19),  namely 

(19)  x2  +  a  -  8x 


is  differentiated  implicitly  with  respect  to  a,  observing 
that  x  =  x(a),  to  give 


(26) 


2x 


dx 

HiT 


+  1 


8 


dx 

Ha 


whence 


(27) 


dx  1 

HiT  "  "  2"(x-4) 


Setting  a  =  12  and  x  =  6  (remember  that  it  is  known  that 
x  =  6  when  a  =  12)  gives 


(24) 


dx 

da 


a  =  12 


1 


Observe  that  Eq .  (21)  did  not  appear  at  all  in  this  method. 

Next,  we  consider  a  case  involving  two  parameters.  The 
equations  are 


(15)  ax2  +  y2  =  25 

x  +  by  =  5 
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and  it  is  desired  to  find  the  behavior  of  the  solutions  in 
the  vicinity  of  a  ■  1 ,  b  ■  3 ,  As  in  our  previous  example,  it 
is  possible  to  express  both  x  and  y  explicitly  as  functions 
of  a  and  b,  since  nothing  worse  than  a  quadratic  equation 
is  involved ;  “however ,  we  shall  avoid  this  approach.  First  of 
all,  the  solutions  when  a  -  1,  b  -  3  are  x  -  -4 ,  y  -  3  and 
x  -  5,  y  -  0.  We  shall  treat  the  case  x  -  -4,  y  -  3. 


(28) 


fx  -  x(a  ,b)  -  x(l  ,3)  +  A (a-1)  +  B(b-3) 
\y  -  y(a,b)  -  y (1 ,3)  +  C(a-l)  +  D(b-3) 


where 


(29) 


A 


£2.  n 

da’  B 


d  X 

db  ’ 


*1 

db 


and  the  derivatives  are  to  be  evaluated  at  a  -  1,  b  -  3.  The 
problem  now  consists  of  computing  A,  B,  C,  D.  Differentiating 
Eq.  (15)  partially  with  respect  to  a  and  b  yields 


(30) 


< 


4  +  b  4  -  o 

o  a  da 


2  ax  ^  +  2y 


ay  _  o 
5F  0 


^x  .  dy 

+  b  +  y 


F6 


When  these  are  evaluated  at  a  -  1,  b 
the  result  is 


-4,  y  -  3, 


(31) 


-8A  +  16  +  6C  -  0 
A  +  3C  -  O 

-8B  +  6D  -  0 

B  +  3D  +  3  -  O. 
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These  equations  are  easily  solved  to  give 

(32)  A-|,  B-  |,  C--^,  D  5 

Hence 

(33)  rx  -  -  4  +  ®  (a  -  1)  •  |  (b  -  3) 

ly  tt  3  -  fy  (a  -  1)  -  £  (b  -  3) 

VI T  .  PERTURBATIONS  APPLIED  TO  DIFFERENTIAL  EQUATIONS 


One  of  the  main  applications  of  perturbation  theory  is  to 
systems  of  equations  consisting  partly  or  wholly  of  differen¬ 
tial  equations.  As  an  example,  we  consider 


(18)  xy  -  1 

yi  -  ax  -  1  x  *  y  -  1  when  t  -  0  . 


We  assume  the  parameter  a  is  small  in  absolute  value.  When 
a  *  0,  the  solution  to  the  above  equations  is 


(34) 


How  should  this  be  modified  when  a  ®  0?  We  acknowledge  that 
x  and  y,  as  well  as  being  functions  of  t,  will  also  depend 
on  the  parameter  a.  Hence,  we  write  x  -  x(t,a),  y  -  y(t,a). 
The  linear  approximations  to  these  functions  in  the  neighbor¬ 
hood  of  a  =  0  are 
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(35) 


(36) 


rx(t,  a)  *  x ( t , 
[y(t,  a)  -  y(t, 

r x ( t ,  o)  -  e”* 
\y(t,  o)  -  e* 


o)  +  Aa 
o)  +  Ba 


where 


(see  Eq.  34)  and 


(37) 


evaluated  at  a  -  0  . 


It  should  be  observed  that  A  and  B  ordinarily  will  be 
functions  of  t.  Our  problem  now  is  to  compute  these  deriva¬ 
tives  . 

The  technique  we  use  is  to  observe  that  x(t,  a),  y(t,  a) 
must  satisfy  Eq.  (18)  identically,  that  is 


(38) 


ay 

ST 


1 


3  X 

ST 


ax 


1  . 


Differentiating  these  equations  partially  with  respect  to 
a  yields 


(39) 


r 


A2 

3  y 

Sa  a  t 


»  2x 
^  3  a  3  t 


3  x 
3"a 


dy 

ST 


3  y  o  x 
3a  ST 


_  a x 

a  Fa  + 


V. 
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We  now  set  a  =  0,  x  =  e-t,  y 
to  obtain 


and 


*1  = 
dt 


(40) 


r 


< 


-t  dB  .  t. 

e  37  +  e  A 


t  dA  -tD 
®  dt  "  G  B 


=  0 


e 


t 


which  are  the  equations  we  must  solve  in  order  to  find  A  and  B. 
It  is  evident  that  our  procedure  so  far  is  the  same  as  it  was 
in  our  previous  examples.  Here  we  must  contend  with  differen¬ 
tial  equations,  and  the  equations  we  must  finally  solve,  i.e., 
Eq.  (40),  are  themselves  differential  equations,  whereas  our 
final  equations  were  algebraic  in  the  other  examples.  Never¬ 
theless,  the  technique  for  finding  those  equations  is  the  same 
in  both  cases. 


Since  Eq .  (40)  is  a  differential  equation,  it  may  seem 
that  no  great  advantage  has  been  derived  by  use  of  the  pertur¬ 
bation  analysis,  but  this  would  indeed  be  a  false  evaluation. 
Equation  (40)  is  linear ,  whereas  the  original  Eq.  (18)  is  non¬ 
linear.  This  conversion  from  nonlinear  to  linear  (quite 
naturally  called  linearization)  is  the  direct  result  of  using 
the  linear  approximations.  It  is  a  decided  advantage  for  the 
home  team  to  have  linear  equations  to  solve. 


VIII.  INITIAL  CONDITIONS 


In  order  to  solve  Eq.  (40),  we  require  initial  conditions 
for  A  and  B.  These  are  obtained  by  using  the  initial  condi¬ 
tions  appearing  with  the  original  Eq.  (18).  There  we  see  that 
x  =  y  =  1  when  t  =  0,  no  matter  what  the  value  o£  a  is. 
Hence,  setting  t  =  0  in  Eq .  (35 )  gives  A  =  0,  B  =  0.  These 
are  the  initial  conditions  to  be  used  in  conjunction  with  Eq. 


(40)  . 

The  solutions  are 

(41) 

|A  = 

t  e"* 

Ib  - 

(1  -  t)  e*  -  1  , 

whence 
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(42) 


-t  .  -t 
-  e  +  at  e 


y  -et  +  a  [(1  -  t)et  -  1] 


These,  then,  are  the  answers  we  set  out  to  obtain. 

Now  it  happens  that  Eq.  (18)  can  be  solved  explicitly  for 
x  and  y  in  terms  of  t  and  a.  The  results  are 


(43) 


x  -  e 


-(1-a) t 


e<l-a>t  ,  a 

1-a 


which,  for  a  -  0,  reduce  to  Eq.  (36).  It  should  be  kept  in 
mind,  however,  that  it  is  very  seldom  possible  to  get  such 
explicit  analytic  solutions.  We  introduce  Eq.  (43)  merely  to 
illustrate  another  point.  First,  we  consider  two  infinite 
series . 


(44)  e  ■  1  +  z  +  ^  +  3"y  + 


valid  for  all  z 


1 


(45) 


1 

-z 


2  3 

1  +  z  +  z  +  z  + 


valid  for  I  z  <  1 


Thus,  by  applying  Eq .  (44)  to  Eq .  (43),  we  obtain 

/  a  c\  -(l-a)t  -t  at  -t  r,  .  (at)** 

(46)  x=ev  -e  e  -e  [  1  +  at  +  —  rrj-  +  ,  .  .  ] 


Similarly,  we  obtain 


(47)  y  -  (1  +  a  +  +  ...]  |e*  [  1  -  at  +  -  •  .  1-  aj, 
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First  consider  Eq.  (46).  The  series  1  +  g f  +  ...  con- 

verges  for  all  values  of  a  and  t.  In  addition,  whenever 
the  product  at  is  small  enough,  the  first  two  terms  of  the 
series ,  i . e . “the  linear  approximation  1  +  at,  can  be  used 
without  excessive  error.  Hence ,  Eq.  (46)  yields 


(48)  x  a  e-t  (1  +  at)  -  e_t  +  at  e-t 


if  at  is  sufficiently  small.  This  is  precisely  the  result  ob¬ 
tained  by  the  perturbation  analysis.  (Compare  with  Eq.  (42).) 
The  present  approach  should  make  it  clear  that,  not  only  must 
a  be  close  to  zero,  but  t  must  be  sufficiently  small  also  in 
order  for  the  perturbation  analysis  to  remain  valid.  In  short, 
the  point  we  wish  to  make  is  that,  when  applying  perturbation 
methods  tc  differential  equations,  it  must  be  kept  in  mind 
that  the  results  are  apt  to  be  valid  only  for  a  restricted 
range  of  the  independent  variable,  not  to  mention  the  restricted 
ranges- of  the  parameters.  This  does  not  prevent  the  method  from 
being  a  very  useful  one. 

Equation  (47)  also  can  be  simplified  under  the  assumption 
that  at  and  a  are  small. 


(49)  y  -  (1  +  a)  [et(l  -  at)  -  a]  -  (1  +  a)  [  e*  -  a(tet  +  1)] 


«  e*  +  a  [e*  (te*  +  1)]  -  e1  +  a  [  (1  -  t)  et  -  U  . 


Once  again  this  is  the  result  given  previously  in  Eq.  (42> . 
Clearly  the  procedure  employed  is  to  discard  all  powers  higher 
than  the  first  power  of  the  small  quantity.  This  automatically 
leads  one  to  the  linear  approximation. 
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IX.  A  TRAJECTORY  PROBLEM 


As  another  example,  we  consider  a  simplified  version  of 
a  fairly  complicated  problem.  A  missile  of  mass  m  is  fired 
at  an  angle  of  elevation  y  with  a  muzzle  velocity  b.  There¬ 
after,  it  travels  in  free  flight  subject  only  to  gravity  and 
to  air  resistance  which  is  proportional  to  the  square  of  the 
velocity . 


x  ~  v  cos  9 


The  equations  of  motion  (obtained  from  Newton's  Law)  are 


(50)  fx 


ax 

ay  ~\[ 


■2  2 

x  +  y 

“  *"2 

x  +  y  -  g 


O 


a 


A 

m 


y  -  o 

b  cos  y, 
b  sin  y- 
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One  of  our  simplifications  will  be  to  assume  that  the  earth 
is  flat  over  the  distance  travelled  by  the  missile.  The  main 
simplification  will  be  to  assume  that  a  is  small,  i.e., 
a  -  0.  The  solution  of  Eq.  (50)  depends  on  three  parameters, 
a,  b,  y,  and  may  be  written  as 


(51) 


rx  -  x  (t,  a,  b,  y  ) 
t y  -  y  (t,  a,  b,  v )  . 


The  time  of  flight,  T,  of  the  missile  is  the  time  re¬ 
quired  for  y,  which  started  at  zero,  to  reach  zero  once  again. 
Hence  T  is  defined  by 


(52)  y (T ,  a,  b, y  )  -  0  . 

When  solved  for  T,  this  yields 

(53)  T  -  T  (a,  b,  y)  . 


The  range,  R,  of  the  missile  is  the  value  of  x  when 
t  «  T.  Hence  R  is  defined  by 


(54) 


R  -  x  (T,  a,  b,  y) 


whence,  in  view  of  Eq.  (53), 


(55) 


R  =  R  (a,  b,  y)  . 
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Thus,  T  and  R  are  functions  of  the  three  parameters,  a,  b,  y. 
When  a  -  0,  b  «  b  ,  y  -  yQ,  it  is  very  easy  to  solve  all  the 

equations  involved  and  find  the  values  of  T  and  R.  We  want 
to  find  out  how  T  and  R  behave  when  the  parameters  are 
perturbed. 

The  linear  approximations  to  Eqs.  (53)  and  (55)  are 


(56)  r T 

Ir 


T (0 ,  bQ,  yQ)  +  Aa  +  B(b  -  bQ)  +  C(  y  -  Y Q) 

R(0,  b0,  yQ)  +  Ea  +  F(b  -  bQ)  +  G(  y-  yQ)  where 


U 


(57) 


9’j' 

7a 

dT 

V5 

dT 


G 


dR 

■  *y 


evaluated  at 
a  -  0 
b  -  b_ 


We  must  compute  the  derivatives  A,  B,  C,  E,  F,  G.  For  this  we 
must  turn  to  Eqs. (53)  and  (54)  which  define  T  and  R 
implicitly.  Differentiating  Eqs.  (52)  and  (54)  partially  with 
respect  to  a,  b,  y,  we  obtain 


r 


(58) 

dy 

dT 

dy  n 

dx 

dT 

dx 

dR 

3T 

7a 

+ 

7a  "  0 

7T 

7a 

+ 

7a  “ 

7a 

< 

0y 

7T 

dT 

7S 

+ 

§f-o 

dx 

7T 

dT 

7 F 

+ 

dx 

dR 

7 ¥ 

Oy 

7T 

dT 

dy 

+ 

5y  "  0 

dx 

7T 

dT 

3 y 

+ 

dx 

7y  “ 

dR 

7y  • 
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X.  THE  KNOWN  SOLUTION 


We  digress  momentarily  to  consider  th.  case  a  -  0,  b  ■  b  , 
y  -  y  ,  and  to  compute  some  quantities  which  we  shall  presently 
need.0  We  let 


(59) 


{ 


X  -  x(t,  O,  bQ,  y0) 
Y  -  y(t ,  o,  bo,  y0) 


and  observe  that  these  functions  satisfy 


(60)  fX  -  0 


{ 


-  g 


t  -  o  (x  -Y  -  ° 
LX  -  b^  cos 


v  ,  Y  ■**  b  sin  y 
'o'  o  •  ( 


These  equations  are  particularly  easy  to  solve.  The  results  are 


{ 


t 

COS 

y 

X  -  b^ 

COS 

V 

0  0 

rt- 

sin 

ro 

1  *2 
-  2  gt 

o 

Y  -bo 

sin 

0^  0 

From  these  we  easily  obtain 


(62) 


< 


To  "  T<0’  bo'  Yo> 


rq  -  R<0’  bo'  yQ) 


2bo  sin  Yc 

g 


b  sin  2 y 
o  'o 

g 
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Hence,  when  t  -  T  ,  we  find 
’  o’ 


(63) 


r  X  -  b  cos  v 
|  o  'o 

l  Y  -  -  b  sin  y  . 

o  'o 


Return  now  to  Eq.  (58).  These  must  be  evaluated  at 
a  -  o,  b  -  bQ,  y  -  yQ,  and  therefore  at  T  -  Tq  also.  Under 

these  conditions,  the  derivatives  dy/dT  and  dx/ST  become  the 
expressions  given  in  Eq.  (63).  However,  there  still  remain 
the  derivatives  ey/6a,  f,y/0b,  etc.,  about  which,  as  yet,  we 
have  no  information.  Hence,  we  let 


(64) 


H 

1 

N 

V 


ix 

ua 

-  H  (t) 

p  -  -  P(t) 

0X 

5 ¥ 

-  M(t) 

«  -  &  -  Oft) 

dx 

^y 

-  N(t) 

w  -  p-  -  W(t) 

d  y 

evaluated  at 


a 

b 


0 

b 


The  evaluation  of  Eq .  (58)  now  gives 


(65) 


b  sin  y  A  +  P(T  ) 
O  '  o  o 

b  sin  y  B  +  o(T  ) 

O  '  o  x  o 

b  sin  y  C  +  W(T  ) 

O  'O  '  o 


V. 


0 

0 

0 


bo  cos  yQ  A  +  H(To) 

bo  cos  yQ  B  +  M(To) 

bo  cos  yQ  c  +  N(To) 


E 

F 

G  . 


It  is  a  very  simple  matter  now  to  solve  this  set  of  simultaneous 
linear  equations  for  A,  B,  C,  E,  F,  G.  Unfortunately,  our 
troubles  have  merely  been  transferred  to  another  place.  We  now 
find  it  necessary  to  evaluate  P(T  ),  Q(T  ),  etc.  These  quanti¬ 
ties  are  partial  derivatives  of  tRe  trajectory  variables  x  and 
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y  with  respect  to  the  parameters  a,  b,  y,  and  presumably  can 
be  obtained  by  differentiation  of  Eq.  (51).  However,  we  do 
not  actually  have  the  explicit  representation,  Eq.  (51).  All 
we  have  is  the  original  set  of  differential  equations  (50) 
which  define  x  and  y.  By  differentiating  Eq.  (50)  par¬ 
tially  with  respect  to  a,  b,y  ,  and  then  evaluating  at  a  -  O, 

b  -  b^,  y  -  y  ,  we  obtain 
o  o 


(66) 


d2H 


< 


dt 


d2M 


dt 


d2N 


dt 


-  XV 

0 

0 


d2P 


dt 


d2Q 

57 


d2W 


dt 


-  YV 

0 

0  , 


where 


(67) 


This  is  a  set  of  linear  differential  equations,  in  contrast 
to  Eq.  (50)  which  is  nonlinear.  We  now  require  initial  con¬ 
ditions  for  Eq.  (66)  in  order  to  solve  them. 


XI •  initial  CONDITIONS 

The  linear  approximations  to  x  and  y  are 


rx  -  X  +  Ha  +  M(b  -  b  )  +  N( y  -  y  ) 

(68)  J 

ly  -  Y  +  Pa  +  0(B  -  bo)  +  W(y  -  yo)  . 


Since  we  want  x  -  y  -  O  when  t  -  0,  no  matter  what  values  a,  b, 
y  have,  we  conclude  that  H>=M-N-P-0*W-0  when  t  -  O. 
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These  are  half  of  the  initial  conditions  for  Eq.  (66) .  From 
Eq.  (68)  we  obtain 


(69) 


r  x  -  X  +  Ha  +  M(b  -  bQ)  +  N(  y  -  yQ) 
1  y  -  Y  +  Pa  +  6(b  -  bQ)  +  W(  y  -  yQ) 


Once  again  we  let  t 


0  and  find 


(70) 


from  which  we 
may  write 


cos  y-  bQ  cos  yQ  +  Hq  a  +  MQ(b  -  bQ)  +  NQ(  y  -  yQ) 
sin  ys  bo  sin  yo  +  P„  a  +  QQ(b  -  bQ)  +  WQ(  r  -  yQ) 

hope  to  get  the  initial  values  of  H,  M,  etc.  We 


(71)  b  cos  y  -  b  cos  [  yQ  +  (y  -  yQ)] 

-  b  [  cos  yQ  cos  (y  -  yQ)  -  sin  Yq  sin  (y  -  yQ)] 

Now  we  are  interested  only  in  the  case  when  y  yQ  is  a  small 
angle.  Hence,  we  may  say 


(72) 


r  cos  (y  yo)  -  1 

s.  Sin  (y  -  yQ)  Z  y  -  yQ  . 


(These  are  the  usual  "small  angle  approximations.")  Equation 
(71)  now  becomes 
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(73)  b  cos  y  st  b 

Next  we  write  b  as  b  + 
—  o 

(74)  b  cos  y  -  [ bQ  + 

-  b  cos 
o 


[cos  0  -  (y  -  yQ)  sin  yQ  )  . 

(b  -  bQ) 

(b  -  bQ) ]  [cos  yQ  -  (r  -  yQ)  sin  yQ 
*0  +  (»  -  bo)  cos  rQ  -  ir  V0)  bQ 


) 

sin 


*o 


wherein  we  have  discarded  the  term  involving  the  product  of  the 
two  small  quantities  (b  -  bQ)  and  (y  -  y  ) .  The  result  in 

Eq.  (74),  of  course,  is  merely  the  linear  approximation  to  b 
cos  y  at  (bQ,  y Q) ,  and  could  have  been  obtained  more  easily 

perhaps  by  the  methods  discussed  earlier.  However,  the 

technique  used  here  is  seen  quite  often,  and  is  itself  worthy 

of  attention.  A  comparison  now  of  Eq.  (74)  and  Eq.  (70)  shows 

that  H  -  0,  M  -  cos  y  .  N  -  -  brt  sin  yn  when  t  -  0. 

o  o  '  o 

A  similar  treatment  of  b  sin  y  leads  to 


(75)  b  sin  y  -  bQ  sin  yQ  +  (b  -  bQ)  sin  yQ  +  (  y-  y Q)  bQ  cos  yQ 


so  that  P  =  0,  0  *  sin  y  ,  W  -  b  cos  y  when  t  -  0.  We  now 

•  o  o  o 

have  all  the  initial  conditions  for  Eq.  (66). 

It  is  a  relatively  routine  matter  to  solve  Eq.  (66)  and  to 

evaluate  the  solutions  at  t  -  T  .  The  results  are 
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H(To)  ■  ■  bo  Sin  2  y 


<rV2 

°  / 

Jo 


~)]t2  +  Z32  dt 


M(To) 


b  sin  2y  2b  2  sin2  y^ 

o  .  M/rr  N  _  o  'o 

- g -  '  N(To) - g - 


(76) 


< 


P(T0) 


Q(T0) 


-T  /2  - 

2  1  -2 1t2  +  7 


2g  I  t 

’o 


dt 


2b  sin2  y  b  2  sin  2  yn 

°  ,  °  ;  W(T0)  -  -2 - - - 2  ,  where 


b  cos  y^ 
(77)  fi  -  -2—^ - ° 


After  some  tedious  computations,  the  integrals  in  H(TQ)  and 

P(T0 )  can  be  evaluated,  all  the  results  can  be  substituted 
in  Eq.  (65),  after  which  Eq.  (65)  can  be  solved  for  A,  B,  C, 
E,  F,  G,  which  in  turn  can  be  substituted  in  Eq.  (56)  to 
yield 


r 


b  3  a 
o _ 

4g  sin  y 


(cos  y  )  in 
o 


(78) 


< 


o  *- 


-  sin  yQ  (2-cos  yQ) 


2  b  cos  y 

+  i  — °~i - -  )  O'  -  yj 


1  +  sin  y 


cos  y 


2  sin  Vc 

g 


(b  -  b  ) 
v  o 
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R  -  R 


b  a  cot  y 
o  ro 


o  — 7^ 

4g 


[(cos  y  )(1  +  3  sin  y  ) 


1  +  sin  y 
'o 


o'  cos  y 


(78) 
Cont . 


9  ,  2b  sin  2  y 

sin  y^(l  -  3  sin^  yo) )  +  ( — g - —  J (t»  bj 


'2b  cos  2  y 

- £J<y- V 


If  b  -  305  ft/sec,  y  -  ir/6  radians,  and  g  -  32.2  ft/sec  , 
then  Eq.  °(78)  becomes  ° 


r  T  -  9.48  +  4320  a  +  0.0311  (b  -  bQ)  +  16.4  (y  -  yQ) 
(79)  J 

Lr  -  2500  .  2160000  a  +  16.4  (b  -  bQ)  +  2890  (y  -  yQ)  . 


Thus  a  missile  fired  with  a  muzzle  velocity  of  305  ft/sec  at 
an  angle  of  elevation  of  30°  in  a  vacuum  (a  -  o)  would  travel 
2500  feet  in  9.48  seconds.  A  missile  fired  in  exactly  the 
same  way  in  an  atomosphere  for  wh^ch  a  -  0.002  would  travel 
minus  1820  feet  in  18.12  seconds.  This  oLviously  ridiculous 
result  should  serve  as  a  warning  against  the  incautious  use  of 
results  obtained  by  a  perturbation  analysis.  _The  trouble  here 
is  that  a  -=  0.002,  while  seemingly  quite  small,  is  not  small 
enough.  It  is  apparent  that  whether  or  not  a  quantity  is  small 
is  a  relative  matter.  The  same  quantity  may  be  either  small 
or  large,  depending  on  the  use  to  which  it  is  put. 

A  rough  rule  which  can  be  used  (though  not  always  safely!) 
in  situations  like  this  is  that  the  perturbation  terms  should 
amount  to  no  more  than  a  few  per  cent  of  the  quantity  being 
computed.  Thus,  in  our  problem,  a  should  be  restricted  to 
about  5  x  10"  ,  while  b  bQ  coulcT  have  an  order  of  magnitude 

of  about  10  ft/sec,  and  y  -  yQ  about  0.05  radians  (about  3°). 

So  long  as  a ,  b,  y  have  values  '’onsistent  with  these  restric¬ 
tions,  Eq.  (79)  may  be  used  to  compute  time  of  flight  and  range. 

i 

\ 
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The  problem  just  completed  is  a  typical  example  of  the 
troubles  encountered  in  a  perturbation  analysis.  Practically 
all  of  these  troubles  stem  from  the  fact  that  the  various 
functions  are  defined  implicitly  and  cannot  be  obtained  in 
an  explicit  analytic  form.  Consequent ly ,  it  is  necessary  to 
employ  implicit  differentiation,  which  usually  leads  to  rather 
formidable  looking  expressions.  On  the  other  hand,  any  equa¬ 
tions  which  must  be  solved  are  linear.  The  perturbation 
method  is  well  adapted  to  finding  quick-but  not-too-dirty 
answers  to  a  wide  variety  of  practical  and  theoretical  prob¬ 
lems  . 


XII.  AVERAGE  OR  MEAN 


Whenever  a  series  of  measurements  of  a  certain  quantity 
(say  the  length  of  a  stick,  for  example)  is  made,  a  collection 
of  numbers  x  ,  r  -  1,  2,  ...,  N,  is  obtained.  Because  of 
errors  in  th£  measuring  instrument,  and  perhaps  other  factors 
beyond  the  control  of  the  person  making  the  measurements,  the 
numbers  x  normally  will  not  all  be  the  same,  but  instead 
will  varyrto  an  extent  depending  on  the  factors  affecting  the 
measurement.  The  question  then  arises  as  to  what  value  should 
be  assigned  to  the  measured  quantity.  Furthermore,  it  would 
be  nice  to  have  some  estimate  of  whatever  error  is  apt  to  be 
in  the  assigned  value. 

Let  x  represent  the  value  to  be  assigned.  Then  x  xr 
is  the  residual  or  the  deviation  of  the  r**1  measurement.  A 
standard  way  to  choose  x  is  to  choose  it  so  that  the  function 


N 


r-1 


will  take  on  its  minimum  value.  E  is  merely  the  sum  of  the 
squares  of  the  residuals  (hence  a  positive  quantity)  and  can 
be  minimized  by  proper  choice  of  x.  By  differentiating 
Eq.  (80),  setting  the  derivative  equal  to  zero,  and  solving  for 
x,  we  find 


(81) 


N 


r-1 
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! 

Thus  the  average  of  the  x^,  is  the  value  which  minimizes  the 
sum  of  the  squares  of  therresiduals .  It  also  has  the  following 
easily  verified  property. 


N 

(82)  ^  (x  -  xp)  -  0 

r-1 


XIII.  VARIANCE 


Although  the  average  certainly  does  not  display  all  the 
information  contained  in  t he  complete  set  x  ,  still,  in 
one  number,  it  gives  us  an  important  fact  abofit  that  set. 
Another  important  number  associated  with  the  collection  of  the 
x^  is  the  variance  which  is  defined  by 

(84)  o2  -  J  ^  (x  ..  xr)2 

r-1 


and  is  merely  the  mean  (or  average)  of  the  squares  of  the 
residuals,  the  residuals  being  taken  with  respect  to  the  mean 
x  of  the  xp.  The  variance  is  a  measure  of  how  the  measured 
values  are  spread  around  the  mean.  If  the  measurements  are 
very  accurate,  so  that  the  measured  values  are  all  close  to 
the  mean,  then  the  residuals  will  all  be  small  and  the  vari¬ 
ance  will  also  be  small.  However,  if  the  measurements  are  not 
very  accurate,  so  that  the  measured  values  sometimes  depart 
widely  from  the  mean,  then  the  residuals  will  be  large  and 
the  variance  also  will  be  large.  Thus,  it  appears  that  the 
variance  is  related  to  the  precision  of  the  measurements  and 
can  serve  as  a  measure  of  that  precision. 

These  two  statistical  quantities,  the  mean  and  the 
variance,  are  the  most  important  ones  used  in  present  day  en - 
gineer ing  design. 
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XIV  COMPUTATIONS  BASED  ON  MEASURED  VALUES 


Very  frequently  it  is  necessary  to  compute  a  quantity  on 
the  basis  of  experimental  data.  For  example,  we  may  measure  the 
side  of  a  square  and  then  compute  the  area  of  the  square.  Since 
there  is  an  error  in  the  measured  value,  then  there  will  also 
be  an  error  in  the  computed  value.  We  may  make  a  large  number, 
N,  of  measurements  of  the  side  of  the  square  and  compute,  for 
each  one,  the  area  of  the  square,  obtaining  thereby  N  values 
for  the  area.  For  each  of  these  two  sets  of  numbers,  length 
of  side  and  area,  we  may  now  compute  the  important  statistical 
quantities,  mean  and  variance.  Clearly,  there  should  be  a  re¬ 
lationship  among  these  numbers. 


More  generally,  consider  the  problem  in  which  we  measure 
and  compute  y  by  y  =  f(x)  =  A  collection  of  measurements 


with  mean  x  and  variance  o  4 
r  x 


leads  to  a  collection  of  com- 

2 


puted  values  yr  =  f(x^)  with  mean  y  and  variance 
linear  approximation  to  f(x)  at  x  =  x  is 


The 


(84) 


y  =  f(x)  +  f'(x)(x  -  x) .  Hence 


(85)  y  -  f(x)  ±  f'(x)(x  -  x) 

r  r 


It  follows  from  this  that  the  mean  of  y  is  given  by 

N  N 

^  yr  =  ^  \  lf(x)  +  f’(x)(xr  -  x)] 

N  *  ' 


=  f  (x)  . 


Thus  the  average  value  of  y  can  be  obtained  by  computing  it 
directly  from  the  average  of  x.  Equation  (85)  now  may  be 
written  in  the  form 


(86) 


y  = 


I 

N 
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(87) 


yr  -  y  -  f'(x)  (xr  -  x) 


and  this  can  be  used  in  turn  to  compute  the  variance  of  y. 

(88)  °y2  -  R  2  <Vr  -  y)2  -  IT  ^  2  (xr  -  I)2 

r-1  r-1 

-  [  f  •  (x)  ] 2  .  ^(xr  ‘  X)2  "  lf'(x>J2  <JX2  ■ 

r-1 


This  shows  how  the  variances  of  the  measured  and  computed  values 
are  related. 

It  must  be  borne  in  mind  that  the  results  just  derived  are 
based  on  the  use  of  a  linear  approximation  and  hence  are  subject 
to  all  the  limitations  implied  by  that  approximation.  Only  in 
the  case  of  fairly  accurate  measurements  can  we  expect  Eqs.  (86) 
and  (88)  to  be  valid. 


XV.  FUNCTIONS  OF  MORE  THAN  ONE  VARIABLE;  COVARIANCE 


More  often  than  not,  a  computed  value  is  based  on  measured 
values  of  several  different  quantities.  Thus,  let  xr,  y  zr 

be2Collections  of  measurements  with  means  x,  y,  z  and  variances 

a  ,  a  ,  a  respectively,  and  let  w  =  f(x,y,z).  The  linear 
x  y  z 

approximation  at  (x,  y,  z)  is 


(89) 


w  w  f(x,  y, 


A 

B 

C 


Of 

£x 

Of 

Zy 

Of 

0Z 


z) 


> 


+ 


/ 


A (x  -  x)  +  B(y  -  y)  +  C(z  -  z) 

evaluated  at 
x  -  x 

y  =  y 
z  *=  z 
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It  turns  out,  just  as  before,  that  w  -  f(x,  y,  z) .  Thus, 
Eq .  (89)  may  be  written  as 


(90) 


wr  w  -  A(xr  x)  +  B(yr  -  y)  +  C(zr  -  z) 


which  leads  to 


N 


(91) 


w 


"FT  ^  tA(xr  " 


x)  +  B(yr  -y)  +  C(zr  -  z)] 


r-1 


9  9  9  9  9  9 

A  o„  +  B  o„  +  C  o  +  2AB  +  2AC  o 


x  y 

+  2BC  o 

yz 


xy 


xz 


where  a  ,  a  ,  a  are  new  quantities  defined  by 
xy  xz  yz 


r 


(92) 


N 


'xy  IT 


]>>r  J)(yr  -  y) 


r-l 

N 


yxz"J  ^  (xr  -  X)  (zr  -  z) 


r-1 

N 


yz  ”  IT 


(yr  -  y)(zr  -  z)  . 


These  new  quantities  are  called  the  covariances  of  x  and  y, 
x  and  z,  and  y  and  z  respectively.  They  give  some  idea 
of  how  the  measurements  of  x,  y,  z  are  related.  A  very  common 
situation  is  to  have  independent  measurements,  that  is  to  say, 
the  value  obtained  by  measuring  one  quantity  is  not  affected 
by,  nor  does  it  affect,  the  value  obtained  by  measuring 
another  quantity.  Thus,  if  x  and  y  are  independent  measure¬ 
ments,  we  may  expect  the  product  (xr  -  x)  (y  -  y)  to  be 
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negative  about  as  often  as  it  is  positive,  and  we  may  also 

expect  its  magnitude  to  be  distributed  fairly  equally  between 

the  positive  and  negative  values.  The  net  result  is  that 

a  should  tend  to  zero  as  the  number  of  measurements  increase, 
xy 

In  fact,  we  might  take  o  0  as  an  indication  that  x  and  y 


xy 

are  independent.  If  all  the  measurements  are  independent, 
all  the  covariances  vanish,  and  (9 i)  reduces  merely  to 


then 


(93) 


w 


.2  2 

A  °x 


+  B 


n2  2 
+  C  o 

z 


XVI.  MATRIX  FORM 


A  very  convenient  way  to  handle  the  variances  and  covari¬ 
ances  is  to  write  them  in  matrix  form. 


(94) 


This  is  called  the  covariance  matrix  of  x,  y, 
(91)  now  may  be  written  as  a  matrix  product. 


(95) 


(A  B  C) 


z.  Equation 
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There  remains  one  obvious  generalization.  We  may  make 
measurements  of  n  different  quantities  x,  y,  z,  .  ..,  and 
use  these  measurements  to  compute  m  other  quantities  u,  v, 
w,  ...  .  Knowing  the  covariance  maFrix  of  x,  y,  z,  we 
would  like  to  find  the  covariance  matrix  of  u,  v,  w,  ... 

Let 


(96) 


n  x  n  matrix 


be  the  covariance  matrix  of  x,  y,  z,  ...,  and 


(97) 


uv 


vw 


uw 


vw 


m  x  m  matrix 
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be  the  covariance  matrix  of  u,  v,  w,  ...  .  Also  let 


(98)  D 


d  U 

3u 

'u 

U  X 

ffy 

u  Z 

dV 

dv 

,■>  V 

0  X 

Jy 

cTz 

3w 

3w 

3w 

7x 

3y 

3z 

■\ 


/ 


m  rows 

n  columns 


be  the  matrix  of  partial  derivatives  of  u,  v,  w,  ...  with  re¬ 
spect  to  x,  y,  z,  ...,  all  these  derivatives  being  evaluated  at 
(x,  y,  z,  ...).  Then  exactly  the  same  sort  of  analysis  as  in 
the  simpler  cases  shows  that 


(99) 


V  -  D  M  D 


where  D  is  the  transpose  of  D 


XVII.  STANDARD  DEVIATION  AND  PRECISION 


Suppose  we  have  a  large  number  of  measurements  xf  of  a 
quantity  x.  Then  the  mean  x  and  the  variance  can  be  com¬ 
puted  in  the  manner  previously  described.  The  number  o 
itself  is  called  the  standard  deviation  of  the  data.  It  is 
customary  to  express  the  precision  of  the  measurements  in 
terms  of  a  (rather  than  o^) .  In  ordinary  circumstances  we 
may  expect  about  50  per  cent  of  the  measurements  to  fall  in 
the  interval  from  x  -  0.67o  to  x  +  0.67 o.  (The  coefficient 


t 
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0.67  is  determined  on  the  basis  of  probability  theory  and 
certain  quite  reasonable  assumptions  about  the  nature  of  the 
errors  to  be  expected  in  measurements.  Different  coefficients 
correspond  to  different  percentages.  Thus,  about  90  per  cent 
of  the  measurements  fall  in  the  interval  from  x  -  1.65o  to 
x  +  1.65o.  It  is  in  this  sense  that  o  determines  the  pre¬ 
cision  of  the  set  of  measurements.) 


XVIII.  APPLICATION  OF  STATISTICS  TO  THE  TRAJECTORY  PROBLEM 


To  illustrate  one  of  the  uses  to  which  these  ideas  from 
statistics  may  be  put,  we  return  to  the  trajectory  problem  and, 
in  particular,  to  one  of  the  results  in  Eq.  (79). 


(100)  R  a  2500  -  2160000  a  +  16.4  (b  -  bQ)  +  2890  (y-  y  ) 


where  bQ  -  305  ft/sec  and  yQ  «  ir/6  radians.  To  simplify  the 
present  discussion,  we  assume  a  »  0  (vacuum  trajectory).  Thus, 


(101)  R  -  2500  a  16.4  (b  -  b  )  +  2890  (y  -  y  )  .  , 

I 

i 

i 

I 

This  expresses  the  range  error  due  to  errors  in  muzzle  velocity 
and  angle  of  elevation.  These  errors  are  unavoidable.  Conse¬ 
quently,  we  cannot  expect  to  get  a  range  of  exactly  2500  feet 
on  any  given  shot.  A  much  more  reasonable  demand  is  to  require, 
say,  that  50  per  cent  of  the  shots  should  fall  within  67  feet 
of  the  target,  i.e.,  from  2433  to  2567  feet.  In  terms  of 
standard  deviation,  this  means  that  we  want  0.67 oR  «  67,  or 
a  2  -  104.  But  K 


(102)  oR2  -  (16. 4)2  ob2  +  (2890) 2  ay2  . 


-  104  - 

1 

I 


I 
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(See  Eq.  (93).  We  assume,  of  course,  that  muzzle  velocity  and 
angle  of  elevation  are  independent.)  This  equation  now  consti¬ 
tutes  a  restriction  on  the  values  of  o.  and  a  ,  which  are 

b 


determined  by  the  precision  of  manufacture  of  the  missile  and 
its  launcher.  Thus,  if  launchers  are  manufactured  in  such  a 


way  that  o  2 


10 


-4 


then  Eq.  (102)  gives  o ^  -  34.1.  This 


means  that  the  powder  charge  (or  whatever  controls  the  muzzle 
velocity)  must  be  measured  accurately  enough  so  that  o  ^  - 
34.1  (or  less)  in  order  that  at  least  the  required  percentage 
of  hits  will  be  made  in  the  target  area. 


XIX.  SOME  APPLICATIONS  OF  PERTURBATION  ANALYSIS  TO 
SATELLITE  PROBLEMS 


Now  that  artificial  earth  satellites  have  become  fact 
rather  than  fiction,  there  are  many  problems  wh’ch  no  longer 
are  of  merely  theoretical  interest  but  are  also  of  great  practi 
cal  importance.  Information  about  the  environment  of  the  satel 
lite  can  be  collected  and  transmitted  back  to  earth  by  instru¬ 
ments  carried  in  the  satellite.  Besides  this,  a  great  deal 
more  information  can  be  derived  from  observations  of  the  path 
type  of  information  and  ways  of  obtaining  it. 

The  complexity  of  the  problems  precludes  a  detailed  analy¬ 
sis  in  this  brief  report.  Our  intent  here  is  to  describe  some 
problems,  indicate  how  perturbation  analysis  is  applied,  and 
exhibit  some  of  the  resulting  equations. 


XX.  GRAVITY 


One  important  purpose  of  artificial  earth  satellites  is 
to  obtain  information  about  the  fine  structure  of  the  gravita 
tional  field  of  the  earth.  This  can  be  done  by  observing  the 
motion  of  a  satellite  which  is  far  enough  from  the  center  of 
the  earth  so  as  not  to  be  affected  appreciably  by  the  atmos¬ 
phere,  and  yet  close  enough  so  that  irregularities  in  the 
gravitational  field  are  still  large  enough  to  cause  observable 
variations  in  the  satellite's  trajectory.  There  is  good 
reason  to  believe  that  the  irregularities  we  are  trying  to  de¬ 
tect  will  be  "relatively  small,"  so  that  perturbation  techni¬ 
ques  should  provide  a  useful  tool  for  finding  the  gross  or 
"first  order"  effects  to  be  expected. 
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XXI.  TWO  BODY  PROBLEM: 


We  consider  first  a  simplified  problem  (without  the 
irregularities  mentioned  above)  which  we  can  solve  analytically. 
This  is  the  standard  "two-body  problem."  We  assume  that  the 
earth  is  a  homogeneous  sphere  of  mass  m£.  As  a  result  of  this 

assumption,  the  force  of  attraction  of  the  earth  on  an  external 
body,  say  a  satellite  of  mass  mg,  is 


(103  ) 


G  mE  mS 


r 


TT 


where  G  is  the  gravitational  constant  and  r  is  the  distance 
from  the  center  of  the  earth  to  the  external  body  (assumed  to 
be  small  enough,  relative  to  the  earth,  to  be  taken  as  a 
point).  Moreover,  the  force  is  directed  toward  the  center  of 
the  earth.  We  take  a  rectangular  coordinate  system  (x,  y,  z) 
whose  origin  is  at  the  earth's  center  and  whose  axes  have  their 
directions  fixed  in  inertial  space.  The  components  of  the 

Fx  Fv  Fz 

force  F  in  the  x,  y,  and  z  directions  are  -  — ,  -  ~,  and  -  — 

respectively,  since  p  p  >  p  are  the  direction  cosines  for 

the  force  direction.  Newton's  Law,  applied  to  the  x-coordinate , 
gives 


(104) 


There  are  similar  equations  for  the  y  and  z  coordinates.  We 
divide  by  and  let  a  *  G  to  obtain  the  equations  of  motion 

of  the  satellite. 


(105) 


z 


V 


ax 

r3 


a  z 


r 


2 

+  z 
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We  let 

(106)  h  «  y£  -  zy  ,  b  -  z*  -  xz  ,  c  -  xy  -  y* 


Differentiation  of  these  gives 


(107) 


dh 

ar 


yz  -  zjl 


ayz  ayz 


0, 


dc 

ar  • 


Hence  h,  b,  c  are  constants .  Furthermore, 


(108) 


hx  +  by  +  cz  -  0  , 


which  shows  that  the  satellite  always  lies  in  the  plane  with 
this  equation.  Inasmuch  as  h,  b,  c  are  constants,  this  plane 
does  not  alter  its  orientation  with  time. 


XXII.  POLAR  FORM 


Now  that  we  know  the  satellite  moves  in  a  fixed  plane, 
we  move  our  coordinate  system  so  that  the  x  and  y  axes 
lie  in  that  plane. 
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The  equations  of  motion 


then  become 


(109) 


It  is  convenient  to  introduce  polar  coordinates  (r,  0)  by 
means  of  the  usual  transformations.  Substitution  in  (109) 
leads  to 


(110) 


r  ©  + 


2  r 


The  second  of  these  equations  can  be  written  as 

(111)  i  ^  (r2  6)  -  0.  Hence 


.  2  ' 

(112)  r  0  =  n  =  constant. 


We  now  can  solve  Eq .  (112)  for  0  and  substitute  in  the 
top  of  Eq .  (110)  to  obtain  a  differential  equation  for  r  in 
terms  of  t. 


(113) 


a 


Presumably  this  can  be  solved  to  obtain  r  as  a  function  of 
t;  however,  it  is  much  more  informative  to  ignore  the  depend¬ 
ence  of  r  upon  t,  and  to  find  out  how  r  depends  on  0. 
This  will  give  us  the  trajectory  of  the  satellite  in  the  plane 
without  specifying  just  where  the  satellite  is  in  that  trajec¬ 
tory  at  any  given  time.  Later  we  can  return  to  answer  the 
question  of  time  dependence.  Hence,  we  differentiate  r  =  r(0) 
twice  with  respect  to  t,  after  which  some  routine  maneuvering 
with  Eqs .  (112)  and  (113”)  leads  to 


! 


t 

I 
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(114) 


The  change  of  variable  r 


-  now  gives 
P 


(115) 


P 


a 


which  is  an  easily  solved  linear  differential  equation. 
The  solution  is 


(116)  p  -  ^  +  k  cos  (9  -  0o) 

n 


where  k  and  9  are  arbitrary  constants  which  depend  ulti¬ 
mately  on  the  initial  conditions. 

From  Eq.  (116)  we  obtain 


(117) 


a  ( 1  -€  2 ) _ 

1  +  €  cos  (0  - 


where 


(118) 


e 


n 


a (1  -  €2) 


When  |€|<  li  Eq.  (117)  is  the  polar  equation  of  an  ellipse 
with  one  focus  at  the  pole.  Thus,  the  trajectory  of  the 
satellite  is  an  ellipse  with  the  center  of  the  earth  at  one 
focus.  The  size  and  shape  of  the  ellipse,  i.e.,  the  semi¬ 
major  axis  a  and  the  eccentricity  e  depend  on  the  initial 
conditions.  —  — 
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The  hitherto  neglected  time  dependence  now  can  be  attacked 
by  substituting  Eq.  (117)  into  Eq.  (112)  to  obtain  a  differen¬ 
tial  equation  for  0  in  terms  of  t.  We  shall  not  do  this  since 
we  do  not  need  the  results  in  what  is  to  follow. 


XXIII.  ORBIT  PARAMETERS 


The  general  solution  of  Eq.  (105)  involves  6  arbitrary 
constants.  To  specify  any  particular  trajectory  and  the  posi¬ 
tion  of  the  satellite  in  that  trajectory,  then,  6  conditions 
are  required.  A  reasonable  choice  is  the  set  of  values  x,  y, 
z,  x,  y,  z  at  some  initial  time  t  (usually  taken  to  be 
t  -  0)  --the  so-called  initial  con8itions.  The  initial  posi¬ 
tion  and  velocity  coordinates  thus  constitute  a  set  of  6 
parameters  which  completely  define  the  Motion  of  the  satellite. 
However,  this  is  by  no  means  the  only  set  of  6  parameters  which 
can  be  used.  Other  commonly  used  parameters  are  the  6  Keplerian 
parameters.  T».o  of  these  specify  the  orientation  of  the  orbital 
plane.  Three  more  specify  the  orientation,  size,  and  shape 
of  the  ellipse  in  the  orbital  plane.  (These  3  are  0  ,  a,  t  .) 
The  sixth  parameter  is  the  time  at  which  the  satellite  passes 
through  some  particular  point  of  its  orbit,  say  the  point 
closest  to  the  focus  (perigee).  The  particular  problem  being 
studied  determines  which  set  of  parameters  should  be  used. 


XXIV.  OBLATE  EARTH 


Unfortunately,  instead  of  being  a  homogeneous  sphere,  the 
earth  is  a  non--homogeneous ,  oblate  spheroid.  It  is  somewhat 
like  a  sphere  with  a  belt  around  the  equator. 
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The  effect  of  the  belt  is  to  add  a  small  component  of  force 
to  the  gravitational  attraction  of  the  sphere.  The  new  force 
of  attraction  now  has  a  component  parallel  to  the  axis  of  the 
earth,  and  consequently  no  longer  is  always  directed  toward 
the  center  of  the  earth.  This  brings  with  it  all  sorts  of 
complications  in  the  motion  of  the  satellite. 


The  equations  of  motion  of  a  satellite  now  become 

+  X  x  f 
(119)  ^  y  =  -  — ^  +  X  y  f 


••  ax 

x  3. 

r 


r 

'■  _  az 
z - 3- 

r 


+  X  z  (f  +  -3-  ) 
r 


where 


„  _  3 (r2  -  5z2) 

1 - 7 - 

2r 


and  X  is  a  parameter  whose  mangitude 


depends  on  the  degree  of  oblateness  of  the  earth.  (For  these 
equations,  the  x  and  y-axes  must  lie  in  the  earth's  equa¬ 
torial  plane  while  the  z-axis  points  in  the  direction  of  the 
north  pole.  The  form  of  the  function  f  and  the  derivation 
of  the  above  equations  of  motion  are  important  matters  which 
we  shall  not  cover  here.)  The  solutions  of  Eq .  (119)  now  will 
depend  on  X 


(120)  x  =  x ( t ,  X)  ,  y=  y(t,  X)  ,  z  =  z(t,  X) 


The  linear  approximations  to  these  at  X  =  0  are 


(121) 

x  =  X  +  M  X ,  y 

=  Y  +  N  X,  z  =  Z  +  P  A 

where 

(122) 

** 

X  =  X(t)  =  x ( t , 

0) 

< 

Y  =  Y(t)  =  y (t , 

0) 

Z  =  Z(t)  =  z(t , 

0)  and 
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(123) 


M 


dx 

d\ 


»-  If 


evaluated  at  X  =  0. 


Differentiation  of  Eq.  (119)  partially  with  respect  to  \ , 
followed  by  evaluation  at  X  =  0,  gives  the  following  differen¬ 
tial  equations  for  M,  N,  P. 


(124)  < 


M 


N 


M  ?X 


(XM  +  YN  +  ZP)J 
(XM  +  YN  +  ZP) 
(XM  +  YN  +  ZP) 


i 


+  XF 

+  YF 

+  Z  (F  +  -2-  ) 

R5 


where  R  -  ~^X2  +  Y2  +  Z2  and  F  «  ^ — ~m5Z  )  .  Now  X,  Y,  Z 

2R ' 

presumably  are  known  functions  of  t  (although  they  may  be 
rather  complicated)  since,  when  A  =  0,  the  equations  of  motion 
reduce  to  the  case  which  we  previously  managed  to  solve  analyti¬ 
cally  to  obtain  the  Kepler  ellipse.  Hence,  the  system  of 
simultaneous  linear  differential  equations  in  Eq.  (124)  can  be 
solved  and  the  results  substituted  in  Eq .  (121)  to  find  the 
actual  trajectory  of  the  satellite.  This  would  indeed  be  a 
formidable  undertaking  on  paper.  Fortunately,  we  shall  be  able 
to  make  use  of  the  equations  in  Eq.  (124)  without  actually 
solving  them. 


XXV.  ROTATION  OF  ORBITAL  PLANE 

We  return  to  Eq .  (106),  namely 


(106)  h  =  y  z  -  z  y,  b  =  z  x  -  x  z,  c  =  x  y  -  y  x. 
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Previously,  when  X  =  0,  we  found  that  h,  b,  c  were  constants. 
When  X  /  0,  however,  there  is  no  reason  to  suppose  that  they 
will  still  be  constants.  Hence  we  write 


(125)  h  =  h(t,  X),  b  =  b(t,  X),  c  =  c(t,  X) 


and  introduce  the  linear  approximations 


h  =  b (t ,  o)  +  H  X 


026)  {  b  =  b  (t ,  o)  +  B  X 


l^c  -  c  (t ,  o)  +  C  X 


where 


r- 


(127)  <  B  -  JX 


I" 


!£. 

ax 


The  equation 


evaluated  at  X  -  0 


(108)  hx  +  by  +  cz  =  0 


still  holds,  even  if  h,  b,  c  are  not  constants.  When  X  =  0, 
Eq.  (108)  is  the  equation  of  the  orbital  plane,  and  h,  b,  c 
(which  are  constant)  determine  the  orientation  of  that  plane. 
When  X  /  0,  Eq .  (108)  is  still  the  equation  of  a  plane,  and  h, 
b,  c  still  determine  the  orientation;  but,  since  h,  b,  c  now 
vary  with  time,  the  plane  is  no  longer  fixed.  We  may  still 
refer  to  it  as  the  orbital  plane ,  since  the  satellite  always 
lies  in  it,  but  we  must  permit  it  to  move  around.  Our  present 
purpose  is  to  find  out  how  it  moves.  This  we  hope  do  to  by 
studying  the  derivatives  H,  B,  C. 
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to  X 

and 

then 

setting 

1 

r 

• 

• 

H 

=  Y 

P 

+  Z  N 

(128) 

< 

B 

=  Z 

• 

M 

+  X  P 

C 

=  X 

• 

N 

• 

+  Y  M 

which 

expresses 

H, 

B,  C 

Y,  Z  and  the  solutions  M,  N,  P  of  Eq.  (124).  Differentiation 
of  Eq  (128)  with  respect  to  time  yields 


(129) 


H  =  YP-YP+ZN-XN 
B  =  X  M  -  Z  M  P-XP 
C  =  X  N  -  X  N  +  YM-YM  . 


In  these,  the  second  derivatives  of  M,  N,  P  can  be  replaced 
through  use  of  Eq.  (124),  while  the  second  derivatives  of  X, 
Y,  Z  are  given  by  X  =  -  a  X  etc.  This  simplifies  Eq. 

(129)  to 


(130) 


H  = 


3YZ 


C 


=  0  . 


It  is  a  simple  matter  to  verify  that  the  point 
(b,  -h,  o)  lies  in  the  orbital  plane  as  well  as  in  the  xy- 
plane.  Hence,  the  line  segment  joining  this  point  to  the 
origin  lies  in  the  intersection  of  these  two  planes.  Let  vi 
be  the  angle  between  the  line  segment  and  the  x-axis.  Then 
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(131)  d  =  -  arctan  £- 

b 


is  one  of  the  two  keplerian  parameters  determining  the  orien 
tation  of  the  orbital  plane.  (The  other  is  the  angle  of  in¬ 
clination  of  the  orbital  plane  to  the  equatorial  plane.) 


(132) 


=  srf(t,  X)  =  «*(t,  o)  +  <J>X, 


•-  (If) 


X  =  0 


From  Eq.  (131)  we  find 


(133)  §£ 


K  ah  .  Sb 

ba"haT 

.2  .2 

b  +  h 


which,  upon  setting  X  =  0,  gives 


(134) 


$  -  - 


boH  -  hoB 
b„2  +  h  2 


Hence 


(135) 


$>  =  . 


b  H  -  h  B 
o  o 

b02  +  h 2 


3Z 

7  ' 


hoX  +  boY 

b  2  +  h  2 
o  o 


3co  Z 

(b7  +  ho2)  R5 
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since  hQX  +  bQY  +  cqZ  =  0  from  Eq .  (108).  Therefore,  differen¬ 
tiation  of  Eq .  (132)  with  respect  to  t  gives 


(136) 


0  = 


3  A  c  Z 
o 

- 2 - T~ 

<bo  +  »o  > 


This  simple  expression  contains  a  large  amount  of 
information,  only  part  of  which  we  shall  endeavor  to  extract. 
The  variable  part  of  d  is  Z^R~^ ,  which  is  never  negative. 
Hence,  vi  never  changes  sign.  This  means  that  is  either 

always  increasing  or  always  decreasing,  depending  on  the  sign 
of  c  .  This,  in  turn,  means  that  the  line  segment  which  de¬ 
fines  0  rotates  in  the  xy-plane  (though  not  uniformly)  and 
the  orbital  plane  rotates  with  it.  The  instantaneous  rate  of 
rotation  is  given  by  Eq .  (136).  Although  we  shall  not  do  it 
here,  it  is  possible  without  much  trouble  to  compute  the 
average  rate  of  rotation  of  the  orbital  plane  from  Eq.  (136). 
this  average  rate  is  easily  observable  in  actual  satellites, 
and,  since  it  is  directly  proportional  to  A,  can  be  used  to 
estimate  the  value  of  A.  This,  of  course,  gives  a  measure 
of  the  earth's  oblateness.  In  this  way,  observation  of  the 
motion  of  artificial  satellites  gives  us  information  about 
the  shape  of  the  earth. 

The  preceding  long-winded  exposition  was  designed  to 
show  how  perturbation  techniques  have  been  applied  to  find  the 
effects  of  the  earth's  oblateness  on  the  orbits  of  satellites. 
The  particular  effect  which  was  studied  was  the  rotation  of 
the  orbital  plane.  There  are  other  effects  also,  all  of  which 
have  been  studied  by  the  same  method  and  are  to  be  found  dis¬ 
cussed  in  various  published  sources. 


XXVI .  SATELLITE  TRACKING 


Although  it  is  both  interesting  and  important  to 
have  the  type  of  information  given  by  the  above  analysis, 
there  are  even  more  important  factors  to  be  considered  in 
the  practical  problem  of  tracking  a  satellite.  To  "track”  a 
satellite  means  to  determine  its  position  (and  perhaps  velocity) 
in  space,  and  also  involves  the  ability  to  predict  future 
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positions  on  the  basis  of  present  knowledge.  The  obvious  way 
to  do  this  is  to  integrate  Eq.  (119),  namely 


x  = 


ax  ,  .  . 

-  — +  X  x  f 

r 


(119)  <y=-“£+Xyf 


r 

az 


3 (r2  -  5z2) 


-J~2  2 2 

r  -  Vx  =  y  +  z 


A  high-speed  digital  computer  can  handle  this  task  both  easily 
and  satisfactorily.  It  is  only  necessary  to  supply  the  machine 
with  initial  values  of  position  and  velocity,  i.e.,  xQ,  y  ,  zQ, 

xq,  y  ,  zQ,  and  with  a  time  interval  At,  after  which  the  machine 

produces  values  of  x,  y,  z,  x,  y,  z  at  the  times  tQ  +  n  At , 

n  =  1,  2,  3,  ...  .  This  all  sounds  very  simple;  unfortunately, 
the  big  problem  still  remains.  It  is  still  necessary  to  make 
the  machine  computations  apply  to  some  particular  satellite 
which  we  wish  to  track.  This  means  that  we  must  properly 

choose  the  six  parameters  (or  initial  conditions)  xQ,  yQ,  zQ, 

•  •  • 

xo,  yQ,  zQ.  Mathematically,  then,  the  tracking  problem  re¬ 
duces  to  the  determinatidri  of  6  parameters. 


In  order  to  determine  the  parameters  we  must,  by 
observing  the  satellite,  make  measurements  of  some  quantity 
which  depends  on  the  parameters.  Then  we  can  compare  the 
measured  values  with  computed  values  of  the  quantity  and 
attempt  to  juggle  the  parameters  so  as  to  bring  the  computed 
values  into  agreement  with  the  measured  ones.  Naturally  there 
must  be  some  mathematical  technique  for  doing  this  juggling. 
The  one  which  is  used  is  the  method  of  least  squares . 


XXVII .  DOPPLER  SHIFT 


Several  different  things  have  been  measured  in 
various  tracking  schemes.  The  emphasis  at  APL  is  on  the 
doppler  shift . 
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Let  the  upper  curve  in  the  diagram  be  the  trajectory  of  a 
satellite  carrying  a  radio  transmitter  operating  at  a  fre¬ 
quency  of  fT  cycles  per  second.  Let  the  lower  curve  be  the 

path  of  an  observer  on  the  earth.  The  observer  moves  rela¬ 
tive  to  the  satellite  path  because  of  the  rotation  of  the 
earth  on  its  axis.  At  time  t,  the  observer  receiver  a  sig- 

g 

nal  from  the  satellite  which  was  emitted  at  time  t  -  — .  S 

c 


is  the  distance  from  the  observer  to  the  position  occupied  by 
the  satellite  when  the  signal  was  emitted,  and  c  is  the 
speed  of  light.  At  a  time  At  seconds  later,  when  satellite 
and  observer  have  both  moved  to  new  positions,  the  situation 
is  as  pictured  in  the  diagram.  The  time  interval  over  which 
the  satellite  has  been  transmitting  its  signal  is  (t  +  At 


S  +■  AS 


)  -  (t  -  §•)  =  At  - 


AS 


c  c 

ber  of  cycles  emitted  is  f 


seconds,  so  that  the  total  num- 


T<*t  -  f  > 


All  of  these  cycles 


are  received  by  the  observer  in  time  At.  Hence,  the  observer 
thinks  the  transmitter  is  operating  at  a  frequency  of  f  = 

-  A Sx 

f T  1  as 

- ^ - =  fT  (1  -  --  Yf)  .  The  amount  of  the  doppler  shift 


is  the  difference  between  the  apparent  frequency  and  the  a.ct- 

f 

T  aS 

ual  frequency,  i.e.,  fQ  -  f^,  =  -  —  .  The  instantaneous 

doppler  shift  D  -  D(t)  now  can  be  obtained  by  letting  At  0. 


(137) 


D 


c  dt 
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It  is  fairly  easy  to  make  very  precise  measurements  of  the 
function  D.  To  be  able  to  compute  D,  we  must  be  able  t)  com- 
dS  ■ "  “ 

pute  -p^  .  This  derivative  is  the  rate  of  change  of  the  dis¬ 
tance  between  satellite  and  observer.  Presuming  that  we  know 
the  position  of  the  observer,  we  will  be  able  to  compute  S 
dS 

(and  hence  if  we  have  a  way  for  computing  the  position  of 

the  satellite.  But  this  is  just  what  the  integration  of  Eq. 
(119)  gives  us,  namely  the  position  of  the  satellite  as  a 
function  of  the  6  parameters.  Hence,  D  itself  is  a  function 
of  those  same  6  parameters.  The  route  may  be  devious  and  the 
expressions  complicated  (fortunately  not  toe  complicated), 
but  the  important  fact  remains  that  D  depends  on  the  parame¬ 
ters  we  wish  to  determine. 


XXVIII.  PERTURBATION  OF  THE  TRAJECTORY 


As  indicated  previously,  the  method  of  least  squares 
is  used  to  find  the  parameters.  All  the  details  of  this  pro¬ 
cedure  are  to  be  found  in  several  APL  reports.  Here  is  it 
merely  cur  intention  to  show  how  perturbation  methods  become 
involved  in  the  computations,  and  to  indicate  the  nature  of 
the  resulting  expressions.  Basically  our  problem  is  to  find 
out  how  the  satellite  trajectory  (which  determines  D)  depends 
on  the  parameters.  The  dependence,  of  course,  is  given  im¬ 
plicitly  in  the  original  differential  equations  of  motion>Eq. 
(119).  What  we  need  for  use  with  the  least  squares  procedure 
is  something  more  explicit.  The  usual  situation  is  that  we 
have  a  rough  idea  of  what  the  parameters  should  be,  and  we  are 
trying  to  get  more  precise  values.  This  is  the  same  as  saying 
that  we  know  approximately  what  the  trajectory  should  be,  and 
that  we  are  trying  to  find  a  better  trajectory  which  presumably 
is  "close"  to  the  estimated  trajectory.  Hence,  we  attempt  to 
find  the  new  trajectory  by  perturbing  the  old. 

Each  of  the  trajectory  variables  x,  y,  z  is  a  function 
of  the  6  parameters  xQ,  yQ,  zQ,  xQ ,  yQ,  zQ.  Each  may  be  differ¬ 
entiated  partially  with  respect  to  each  parameter.  There  are 
18  such  derivatives.  We,  of  course,  are  interested  in  these 
derivatives  when  they  have  been  evaluated  at  xQ  =  x^ ,  yQ  =  y^  , 
etc .  Let 
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(138)  J  W 
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dx 

d  X 

o 

d  x 

o 

dz 

dx 

o 


evaluated  at  x  =*  x  ,  etc  , 
o  co 


These  are  three  of  the  18  derivatives.  (The  first  zero  sub¬ 
script  on  xqo  means  that  it  is  an  initial  condition;  the 

second  subscript  denotes  a  particular  set  of  initial  condi¬ 
tions.)  The  linear  approximations  to  x,  y,  z  at  x^-y^  >  * 

Xoo-  *00’  *00  are 


I 


terms  involving  the  other 

1  15  partial  derivatives 

1 

I 
I 

♦  *  * 

where  x  ,  y  ,  z  are  the  trajectory  coordinates  computed  for 

the  special  case  x  -  x  ,  etc.  We  shall  content  outselves 

o  00 

with  finding  just  one  of  the  18  simulataneous  equations  which 
can  be  solved  to  obtain  the  18  derivatives. 

Differentiate  the  first  equation  in  Eq.  (119)  par¬ 
tially  with  respect  to  xq. 


X  -  X  +  «  (xo  -  xoo)  + 


(139)  {  y  - 


z  = 


y  *  w  <x0  -  xoo>  + 


Z  +  U  <*o  -  xoo)  + 


I 


I 


(140) 


d2  /dx  \  Jl  dx 

(d^)-  '  *  p  ^  “ 


X  f 


d  X 

d  X 


3x  r 

-  —5  >x 

dx 
d  X 

+  y 

-*1  +  z  ■ 
d  X  < 

r 

O 

O 

faf  dx 

dy 

df  dZ 

{**  *xo 

+  — 

by 

sxo 

dz  dxo 

dZ 


dx 


o 


Now  evaluate  at  x  =  x  ,  etc . 

o  00 
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dt 


*  *  *  ,  * 

Qty  W,  u,|  .X, 


where  f 


*  _  3  [r2  -  5<z*)2] 
2r7 - 


and  now  r 


-J ,  *N2  (  *x2  ,  * , 2 

=  V (x  )  +  (y  )  +  (z  ) 


Also  f 


♦  =  (M) 

x  \3x/ 


evaluated  at  x  =  x 


z  =  z 


There  are 


17  more  equations  similar  to  this  one  which  can  be  solved  for 
Q,  W,  U,  etc.  This  sounds  like  a  nearly  impossible  task,  but 
fortunately,  in  terms  of  the  capabilities  of  modern  electronic 
digital  computers,  it  is  a  rather  simple  one. 


Once  Q,  W,  U,  etc.  are  obtained,  then  Eq .  (139)  is 
the  desired  explicit  representation  of  the  trajectory  in  terms 
of  the  parameters.  With  this  we  conclude  this  illustration  of 
some  of  the  problems  involved  in  satellite  tracking. 


Each  time  a  satellite  passes  near  a  receiving  station, 
a  measurement  of  the  doppler  shift  can  be  made.  By  using  several 
receiving  stations  and  making  measurements  over  long  periods  of 
time  (weeks,  months,  or  even  years),  a  large  amount  of  doppler 
data  can  be  accumulated.  It  would  be  eminently  reasonable  to 
suppose  that  out  of  all  this  data  we  should  be  able  to  extract 
very  precise  estimates  of  the  orbit  parameters,  the  precision 
being  limited  only  by  the  unavoidable  "noise"  in  the  data.  We 
would  expect  the  orbit  parameters  to  approach  constant  values; 
however,  this  is  not  at  all  the  case!  The  parameters  computed 
from  the  data  of  one  week  will  be  different  from  those  computed 
from  the  data  of  the  preceding  week.  From  week  to  week  the 
parameter  values  will  change  or  drift.  This  peculiar  behavior 
is  caused  by  errors  in  the  values  of  a  and  X  used  in  the 
equations  of  motion,  Eq.  (119).  The  number  a,  for  example, 
is  proportional  to  the  mass  of  the  earth,  which  is  known  to 
an  accuracy  of  only  about  1  part  in  10,000.  The  number  X  is 
known  even  less  accurately.  On  the  other  hand,  doppler  shift 
measurements  can  be  made  with  accuracies  up  to  1  part  in  108( 
which  is  several  orders  of  magnitude  better  than  our  knowledge 
of  a  and  X.  Of  course,  the  satellite  is  not  afflicted  by 
this  lack  of  knowledge.  Its  motion  is  governed  by  the  exact 
values  of  a  and  A.,  whereas  we  poor  mortals  are  compelled 
to  use  more  or  less  inaccurate  estimates.  This  means  that 
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the  computed  doppler  shift  will  inherit  the  errors  in  a  and 
,\  and  transmit  those  errors  to  the  computed  values  of  the 
orbit  parameters. 

Since  the  drift  of  the  orbit  parameters  is  directly 
related  to  a  and  A.,  then  we  should  be  able  to  get  improved 
estimates  of  a  and  A.  by  measuring  the  amount  of  drift. 

This  is  one  of  the  results  to  be  expected  from  satellite  track¬ 
ing.  Here  again,  perturbation  analysis  is  an  appropriate  tech¬ 
nique  for  handling  this  complex  problem. 

There  is  one  more  set  of  parameters  which  we  wish 
to  consider.  This  is  the  set  of  3  parameters  which  define 
the  location  of  the  receiving  station,  e.g.,  longitude, 
latitude,  and  distance  from  the  center  of  the  earth.  Hereto¬ 
fore,  we  have  considered  these  to  be  known.  By  using  re¬ 
ceiving  stations  of  known  position,  we  can  eventually,  as  in¬ 
dicated  above,  very  accurately  track  a  satellite.  As  pointed 
out  previously,  this  means  that  we  can  accurately  predict  its 
future  positions.  If  now  we  make  doppler  measurements  at  a 
receiving  station  whose  position  is  not  known  very  well,  then 
we  can  employ  exactly  the  same  techniques  used  in  tracking  to 
determine  the  3  parameters  of  the  receiving  station.  Of  course, 
this  problem  is  simpler  since  only  3  parameters  are  involved. 
This  is  known  as  satellite  navigation .  It  is  potentially  a 
very  accurate  navigation  sc heme T  In  this  particular  applica¬ 
tion,  too,  perturbation  analysis  eases  the  labor  required  in 
the  computations.  It  is  to  be  especially  noted  that  an  ade¬ 
quate  satellite  navigation  scheme  depends  primarily  on  a  good 
tracking  scheme. 


XXIX.  BRIEF  COMMENTS  ON  PERTURBATION  ANALYSIS 


In  common  with  many  other  useful  procedures,  the 
id*  as  behind  perturbation  analysis  are  quite  simple.  The 
application  of  these  ideas  to  particular  problems  leads  to 
woi k  of  varying  degrees  of  difficulty.  The  principal  problem, 
of  course,  is  the  evaluation  of  derivatives.  The  difficulties 
arise  because  it  is  usually  necessary  to  resort  to  implicit 
differentiation  to  obtain  the  desired  derivatives.  Further¬ 
more,  there  may  be  several  intermediate  variables  to  be  con¬ 
sidered-  that  is  to  say,  the  variables  we  are  after  may  be 
related  implicitly  to  certain  other  variables  which,  in  turn, 
arc*  related  implicitly  to  the  basic  variables  appearing  in 
the  mathematical  formulation  of  the  problem.  Most  of  the 
problems  in  satellite  work  are  of  this  type.  As  partial  com¬ 
pensation  for  these  troubles,  perturbation  analysis  leads  to 
linear  equations. 
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APPENDIX:  ELEMENTARY  ASPECTS  OF  MATRIX  ALGEBRA 


A  matrix  is  a  rectangular  array  of  numbers. 


The  above  example  is  a  matrix  with  3  rows  and  4  columns,, 
hence  a  total  of  12  elements.  Two  matrices  are  equal  if  and 
only  if  they  have  the  same  shape,  i.e.,  same  number  of  rows 
and  same  number  of  columns,  and  have  their  elements  in  the 
same  order.  The  following  three  matrices  are  not  equal. 

(l  a\  /l  2  3\  /l  3  2 

\3  4/  \3  4  5/  \3  4  5 

The  purpose  of  a  matrix  is  to  permit  a  colit ction 
of  numbers  to  be  handled  as  an  entity.  The  ability  to  do  so 
turns  out  to  be  extremely  useful. 


Addition - The  sum  of  two  matrices  is  defined  if 

and  only  if  the  matrices  have  the  same  shape  In  that  case, 
the  sum  is  obtained  by  adding  corresponding  elements  together. 


Multiplication - If  A  and  B  are  matrices,  then 

the  product  AB  is  defined  only  under  the  following  condition: 
The  number  of  columns  in  the  left  hand  factor  A  must  be  the 
same  as  the  number  of  rows  in  the  right  hand  factor  B,  When 
this  condition  is  satisfied ,  then  the  elements  of  the  product 
AB  can  be  computed  by  a  procedure  which  sounds  complicated 
but  is  quite  easy  to  carry  out.  The  element  in  the  mill  row 
and  nth  column  of  AB  is  obtained  by  multiplying  the  elements 
of  the  mth  row  Qf  A  by  the  corresponding  elements  of  the  nl“ 
column  of  B  (first  with  the  first,  second  with  the  second, 
etc.)  and  adding  the  products  thus  obtained. 
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BA  ifi  not  dofined.  It  will  bo  notod  that  the  pro¬ 
duct  of  two  matrices  haw  the  same  number  of  rows  as  tho  loft 
factor  and  tho  same  nuabor  of  coluanw  aa  the  right  factor. 


Vector a - A  aatrix  with  only  one  coluan  ia  uaually 

called  a  coluan  vector.  A  aatrix  with  only  one  row  ia  uaually 
called  a  row  voctor. 


Linear  Equat ions - Siaultanooua  1  inoar  oquationa 

can  oaaily  bo  writ ton  in  a  aatric  fora. 

3x  ♦  2y  •  1 
7x  ♦  5y  -  3 


Let  C  -  (J  bo  tho  aatrix  consisting  of  tho  coofficionta 

of  the  unknowns. 

Lot  X  -  (*\  be  tho  coluan  voctor  whoso  elements  arc  tho  un- 

v  y  / 

Unowns.  L  ♦  K  -  bo  tho  coluan  voctor  whoso  oloaonts  arc 
the  constant  terms  in  tho  equation.  Thon 


Unit  Matrix - A  square  aatrix  is  one  which  has  the 

same  number  of  rows  as  coluans.  This  number  is  callod  tho 
order  of  the  square  matrix.  A  very  special  square  matrix  is 
the  unit  matrix  1^  whoso  elements  are  all  zero  except  for  those 
on  the 'main  diagonal,  all  of  which  are  one. 
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1 

0 


Nocond  order 


0 

0 

1 

0 


fourth  order 


SoootlNos  a  eubecript  le  used  to  deelgnate  the  order,  o.g., 
I..  If  A  le  Any  Matrix  and  I  la  a  unit  Matrix  of  the 

appropriate  order,  then  A!  •  A  and  1A  -  A. 


/ 1  2  3\/ 1  0  0  \ 

13  4  iJ[ o  i  :)’ 


1+0+0  0*2*0  0*0*3 


3+0+0  0+4+0  0+0+5 


2 

4 


and  let  B 


Inverso  of  a  Matrix - Let  C 


-2 

3 


Then  BC 


■u 

■U  IMS  l)  •( 

-  ( 


■  (?  i) 


as  before, 


15-14 

-21+21 


10-10  \ 
-14+15  1 


0) 

H 


I  . 
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Similarly,  CB  •  I  It  Is  apparent  that  the  Matrix  B  ha*  a 
apodal  relationship  to  the  Matrix  C.  B  Is  said  to  be  the 

Inverse  of  C,  and  It  written  B  -  C~*.  Of  course,  It  would 

be  equally  proper  to  say  that  C 

C  -  B’1.  This  leads  to  the  state 


Is  the  Inverse  of  B,  l.e., 
sent  <C'1)"  -  C. 


In  general,  whenever  two  square  Matrices  H  and 
M  (of  the  ease  order)  satisfy  the  condition  HM  •  MH  -  1 , 
then  each  Is  the  Inverse  of  the  other.  Thus, 


H  - 


1 

a 

a 


and 


are  inverses  of  each  other  since 


HM  - 


-4+1+3 

-7+2+6 

-11+2+9 


-  I 


and  (slHilarly)  MH  >  I . 

Not  all  (square)  Matrices  have  Inverses.  For  ex¬ 
ample,  A  -  has  no  Inverse.  To  show  this,  let  B  * 

(W  z)  be  Rn  arbitrary  «atrlx  of  order  two.  Then 


AB  - 
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and  It  la  clear  that  AB  can  never  be  equal  to  1  •  I  q  no 

aatter  what  valuea  are  given  to  w,  x,  y,  x.  Matrices  without 
lnvereee  are  said  to  be  singular;  Matrices  with  inverses  are 
nonsingular.  The  deteralnant  of  a  square  aaTrTx  is  the  detwr- 
■inant  whose  elsaenls  ere  the  satrix  eleaents  In  exactly  the 
saae  order.  The  condition  for  a  aatrlx  to  be  nonalngular  is 
that  Its  detorainant  be  different  froa  sero. 

The  problea  of  finding  tho  inverse  of  a  aatrlx  is  a 
vory  laportant  one;  however,  it  will  not  be  discussed  here. 

The  iaportance  of  a  aatrlx  inverse  perhaps  can  be 
eat  lasted  by  considering  the  following  exsaple.  The  set  of 
linear  equations 


4x  ♦  y  ♦  a  •  5 
7x  ♦  2y  ♦  2s  -  -1 
llx  ♦  2y  ♦  3x  -  3 


can  be  written  in  the  aatrlx  fora  as  HX  -  K,  where  X 

/5\  (A  1  1\ 

K  -  (—11  »  and  H  -  7  2  2  is  the  aatrlx  considered 

\  3/  \11  2  3/ 


•(:)• 


previously  with  the  inverse  li 


-1 


-1 


2 

1 

i-8 


-1 

1 

3 


Multi¬ 


ply  both  sides  of  HX  -  K  by  H  to  obtain 


H"1  HX  -  IX  -  X  -  H’1  X 


which  says  that  the  solution  of  the  aatrlx  equation  is 


X 


K 


Hence  x  -  11,  y  -  1,  z  -  -  40  is  the  solution  of  the  original 
set  of  equations.  This  is  easily  verified  by  substitution  or 
even  by  solving  the  equations  in  some  other  way. 
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Tran* pone ,  8vMMQtrlc  Matrlcea— The  tranapoae  of  a 
Matrix  A  It  the  Matrix  obtained  by  interchanging  rowa  and 
coluana.  We  Mhall  denote  the  tranapoae  of  A  by  A. 


The  tranepone  of  a  row  vector  le  a  coluan  vector,  and  con- 
veraely.  A  aatrlx  which  la  equal  to  lta  own  tranepoae  la 
aald  to  be  ayaetrlc . 


8  - 


-4 

5 

1 


9  la  ayaaetrlc . 
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I  .  INTRODUCTION 


In  considering  adaptive  method*  and  devices,  it  would 
be  convenient  to  know  what  is  meant  by  the  term  adaptive .  If 
one  attempts  to  distinguish  the  adaptive  from  the  non-adaptive , 
he  may  have  great  difficulty  in  determining  the  dividing  line, 
and  more  trouble  in  getting  agreement  with  his  conclusions. 

We  consider  one  method  or  device  more  adaptive  than  another, 
to  the  extent  that  it  produces  satisfactory  results  for  a 
significantly  greater  variety  of  conditions. 

Some  people  refer  to  a  device  as  adaptive  if  it  in 
some  way  detects  and  compensates  for  inadequacies  in  its 
operation.  Others  consider  such  a  device  as  demonstrating 
active  adaptation  and  refer  to  devices  without  this  feature, 
but  operating  satisfactorily  over  a  wide  range  of  conditions, 
as  having  passive  adaptation.  Some  people  seem  to  consider  a 
device  adaptive  only  if  its  operation  cannot  be  understood; 
we  see  no  merit  in  such  views. 

Evolution  has  tended  to  produce  living  organisms  which 
are  much  more  adaptive  than  the  products  of  human  technology. 
Our  understanding  of  the  adaptive  principles  used  in  nature 
is  far  too  meager  to  yield  easy  application  to  engineering  de¬ 
signs.  Many  able  people  are  attempting  to  discover  the  methods 
used  by  central  nervous  systems,  but  the  problems  are  still 
very  formidable. 

In  discussing  adaptive  methods,  we  shall  consider  four 
topics:  1<)  adaptive  features  of  standard  techniques;  2)  adap¬ 

tive  filters,  designed  to  handle  a  variety  of  input  signal  and 
noise  characteristics;  3)  compensation  for  unknown  transfer 
functions;  4)  compensating  for  noise  correlated,  in  an  unknown 
way,  with  the  remaining  input. 


II.  ADAPTIVE  FEATURES  OF  STANDARD  METHODS 


Before  considering  complex  methods  design 
adaptivity,  it  would  be  well  to  notice  adaptive 
some  methods  very  widely  used  but  not  generally 
particularly  adaptive.  These  are  the  closed  lo* 
filter,  and  the  bang-bang  servo. 


o  increase 
of 

fVic  v  ^ar 


Closed  Loop  Methods - These  methods  are 

variety  and  complexity  in  living  organisms,  in 


lA  in  c  'ii.  a-  is 

«?•'?  is yji  i 


132  - 


\ 


f 


w**, 


•f; 

M.MMUf 


fM|  MMM  MOM  Ml 

AMOCO  MVUCI  lAlOIAVOftV 
mvN  immo  «*m*« 


of  activity.  One  of  the  achievements  of  cybernetics  was  its 
recognition  of  the  powerful  use  of  closed  loop  methods  in 
life  processes.  Technology  made  little  use  of  closed  loop 
methods  until  nearly  the  present  day,  and  even  now  the  closed 
loop  is  foreign  to  the  thought  of  most  people,  even  scientists 
and  engineers. 

The  first  clear  cases  I  can  find  of  closed  loop  methods 
are  in  computation  -  Newton's  methods  of  successive  approxi¬ 
mation  for  finding  a  square  root  of  a  number  or  a  zero  of  a 
polynomial . 

These  computation  methods  illustrate  some  Interesting 
features  of  closed  loop  systems.  Consider  computing  the 
square  root  of  2.  There  is  a  classical  method  of  doing  this, 
Involving  repoated  processes  similar  to  long  division  but 
more  complex,  and  if  one  mistake  is  made  the  answer  may  be 
grossly  wrong.  Newton's  simple  method  involves  repeated  long 
division,  is  easily  remembered,  and  is  virtually  foolproof. 

The  idea  is  to  guess  a  square  root,  divide  it  into  the  wunber, 
and  average  the  divisor  and  quotient  for  the  next  approxima¬ 
tion.  Thus,  1  guess  1,  divide  it  into  2,  and  get  2.  'The  next 

guess  is  i(l  +  2)  -  1.5,  giving  1.33?r*  as  quotient,  1.416  for 
the  next  guess.  At  the  next  cne  gets  1.41421.  Repeating 

the  process  would  give  12-fi.guje  accuracy.  An  error  at  any 
step  but  a  final  unchecked  one  would  not  affect  the  final  re¬ 
sult,  only  lengthening  the  computation.  This  tendency  to  cor¬ 
rect  eriorr.  is  characteristic  of  closed  loop  methods,  as  is 
the  precision  obtainable  with  simple  operations,  and  to  some 
g/. cent  the  nulling  of  the  difference  between  output  and  input;. 
Since  the  square  root  can  be  determined  even  by  a  man  who  for¬ 
gets  the  classical  method,  the  technique  also  shows  that  lower 
quality  components  can  be  used  than  in  the  open  loop  method. 

A  closed  ) <>'•!<  method  of  finding  the  root  of  a  polynomial 
f(x)  =-  0  ifr.  to  select  a  trial  xQ,  evaluate  f (xQ)/f ' (xQ) ,  and 

choose  at.  the  next  approximation  x,  *=  x„  -  f(x„)/f '  (xA) .  When 

this  method  worka,  it  is  far  easier  to  get  precise  results  than 
with  formal  algebraic  methods,  particularly  if  the  polynomial 
is  of  more  than  the  fourth  degree.  Sometimes  the  method  fails, 
either  because  f ' (x  )  is  zero  or  because  successive  approxima¬ 
tions  diverge.  The  tendency  of  instability  of  the  process  can 
usually  be  alleviated  by  suitably  taking  account  of  f" (x), 
which  is  analogous  to  the  use  of  lead  or  derivative  terms  for 
stabilizing  servo  systems.  The  classical  servo  system  was 
Watt's  steam  engine  governor,  first  analyzed  mathematically 


-  133 


'.’X*  ON  MONM 

AffUfO  WflfSICS  !AfO«ATO«V 

HVM  MMO 


by  Clark  Maxwell  about  one  hundred  years  ago,  leading  to 
Routh'n  study  of  stability  of  servo  systems. 

The  simplest  and  most  widely  studied  closed  loop  system 
is  the  servo  or  inverse  feedback  system,  sketched  in  Fig.  1A. 


Fig.  1A 


Fig.  IB 


The  output  is  produced  by  the  operator  F,  a  linear 
transfer  function,  operating  on  the  'error  signal"  X_  -  Xq. 
Solving  the  relation 


X 


0 


F(Xj 


for  the  transfer  function  Xq/Xj  ,  obtain 


X0  F 

F  +  1 


From  this,  it  is  hard  to  see  any  advantage  for  the 
closed  loop  system.  If  for  some  real  frequency  to,  F(oo),  the 
system  is  obviously  unstable;  and  if  the  system  is  stable  it 
is  equivalent  to  an  open  loop  system  with  transfer  function 
F/(F  +  '.)  as  shown  in  Fig.  IB. 

The  closed  loop  system  can  be  made  stable  by  proper 
design,  bo>VHver,  ar  d  may  be  very  superior  to  the  open  loop 
system  if  we  consider  departure  from  the  ideal  as  in  FJg.  1C 
or  Fig.  ID. 


134 


mt  iohni  won  si  uMrvMUTY 

applied  physic  lABQtATOtv 

Kt VII  V«NO  ttJjmAMD 


Fig.  1C  Fig.  ID 


Assuming  G  a  linear  transfer  function  not  known  and  fixed 
in  advance,  ve  find  in  the  closed  loop  system 


(closed  loop) 


(closed  loop),  while 


(open  loop) 


Variations  in  G  a„:d  Z  affect  the  output  directly 
in  the  open  loop  system,  much  less  strongly  in  the  closed  loop 
(negligibly  for  FG  sufficiently  large).  When  the  operator  G 
is  non-linear,  producing  distortion,  the  results  are  similar. 

The  possibility  for  instability  of  the  closed  loop  is  increased 
by  the  variations  we  have  consirared,  but  in  general  with  proper 
design  the  transfer  functior  xrom  input  to  output  can  be  made 
much  less  dependem  on  these  variations  than  is  that  of  an 
open  loop  system  note  also  that  the  open  loop  is  badly  un¬ 

stable  if  Z  i  J  produced  by  a  constant  Z^O,  whereas  the 
closed  loop  -.an  be  made  stable  against  such  disturbances.  For 
any  reasonable  definition  of  "satisfactory  performance"  the 
cio.-od  loop  system  is  satisfactory  over  a  much  wider  range  of 
permissible  G  and  Z. 

A  linear  filter  has  an  adaptive  feature  if  satisfactory 
performance  is  defined  in  terms  of  mean  square  deviations, 
since  such  performance  depends  only  on  the  first  and  second 


XQ  “  Z  •  FG(Xj  -  XQ) 


Xq  FG  *  Z/Xj 


Xj  FG  + 


v  .  FG  v  .  Z 

x0  ‘  FCTTT  xl  +  PS"  +  I 


x0  “  re—rr  XI  •  z 
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moments  of  the  signal  and  noise,  and  is  thus  unaffected  by 
any  further  details  of  the  probability  distribution. 

A  bang-bang  servo  system  is  one  in  which  there  is  a  so- 
called  bistable  element,  sometimes  referred  to  as  a  switch 
cr  relays  This  element  produces  the  maximum  possible  numeri¬ 
cal  output  with  the  same  sign  as  the  input.  In  a  missile  auto¬ 
pilot,  the  output  might  be  control  surface  position  or  rate. 

In  such  a  system  the  gain  may  be  very  large,  but  positive 
instability  is  prevented  by  the  effective  gain  of  the  bistable 
element  being  reduced  as  the  system  nears  instability.  Such 
systems  have  been  studied  extensively  in  recent  years  at  APL 
and  elsewhere.  A  practical  problem  with  these  systems  is  the 
excessive  dissipation  of  power. 

The  super-regenerative  detector  has  some  points  of 
similarity  with  the  bang-bang  servo,  and  demonstrates  some 
vaguely  similar  features. 


III.  ADAPTIVE  FILTERS 


You  have  learned  something  of  the  problem  of  filtering 
signal  from  noise,  and  have  learned  about  Wiener's  filtering 
theory,  giving  minimum  mean  square  error  in  equilibrium,  for 
stationary  signal  and  noise  with  known  spectral  densities. 

We  now  consider  the  problem  of  devising  a  filter  to  give 
satisfactory  performance  for  a  variety  of  input  characteristics. 


Polynomial  Signal - Suppose  that  the  signal  is  a  poly¬ 

nomial  in  time,  of  degree  n,  but  is  not  otherwise  restricted. 
Then  the  filter  must  follow  such  a  polynomial  perfectly  to 
avoid  unbounded  errors.  Writing  the  signal  as 


x(t)  "  cQ  +  jt  t  +  ^  t2  +  .  .  .  +  Jf.  tn 


we  see  that 


x  = 


c.  c„  c 

C-,  +  — —  +  — jr  +  ..  +  — 

0  P  P  P11 


c  +  c  ,  p  + 
n  n-1  * 


•  •  •  • 


+  cxP' 


n-1  ,  n 
+  cQp 


n 
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The  steady  state  error  in  following  x  is 


[x  -  x  F(p))  p_o  -  [  X  |l  -  F(p^  i  pK)  . 


A  necessary  and  sufficient  condition  for  zero  steady  state 
error  is  that 


11.  -  0  . 

p->0  p 


Writing  F  in  the  form 

aQ  +  ai^p  +  a2p2  +  .  .  . 
bD  +  blP  +  b2pz  +  . . . 


we  must  have  =  bQ,  a^  =  b1»  . an  “  bn>  witb 


1  -  F  = 


bn+l  ~  an+l 


n+1  /.  \  n+2 

p  +  (bn+2"an+2)P  - 

T 


br»  +  b1p  +  b„p  +  .  .  .  . 


If  the  filtering  is  accomplished  with  a  servo,  the  forward 
transfer  function  G  of  the  servo  is 


2 

F  aQ  +  a^p  +  a2p  +  . . . . 

^  ^  bn+l  "  an+ljpn+1  +  ^bn+2  “  an+2j  + 


The  forward  transfer  function  in  a  servo  filter  must  integrate 
once  more  than  the  degree  of  the  polynomial . 
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If  the  noise  is  stationary  with  known  spectral  density 
0('.u),  one  can  select  the  coefficients  of  F  subject  to  the 


restriction  that  (an,  a. 


0’  "1*  -r  f  n 

make  the  mean  square  error,  J  \  F(oo) 


a„ )  -  (b 
2 


0’ 


b„)- 


which 


Q(oo)du),  as  small  as 


desired,  at  the  expense  of  large  transient  errors. 


The  problem  of  designing  a  time-varying  filter  without 
transient  or  steady-state  error  in  following  a  polynomial  of 
given  degree ,  and  minimizing  the  mean  square  error  due  to 
stationary  noise,  has  been  solved  theoretically.  The  solution 
of  this  problem  in  practice  appears  to  be  unduly  difficult, 
and  perhaps  of  little  value  in  view  of  the  artificiality  of 
the  assumed  signal  characteristics. 


The  theory  can  be  easily  extended  to  consider  signals 
consisting  of  a  stationary  component  of  spectral  density 
$(<h)  plus  any  polynomial  of  degree  n.  The  optimal  constant- 
coefficient  filter  is  that  which  minimizes 


1 


dco 


by  choice  of  F  among  those  transfer  functions  satisfying 
the  restrictions  (a^,  a^,  ..  an>  *  (bQ,  b^,  ..  bn).  We  note 

that  the  filter  described  by  Hanson  is  optimal  not  only  for 
the  noise  and  signal  spectral  densities  assumed,  but  also 
for  cases  in  which  the  signal  has  any  linear  function  of  time 
as  an  additional  component. 


The  Minimax  Filter - The  minimax  principle,  acting  to  I 

minimize  the  maximum  expected  loss,  has  many  advantages, 
whether  one  is  dealing  with  alert  enemies  or  intractable 
nature.  The  approach  just  mentioned  is  minimax  for  signals 
which  may  include  any  n^*1  degree  polynomial,  but  have  a  re¬ 
mainder  with  known  spectral  density. 

A  useful  class  of  signals  to  consider  is  that  in  which 
the  spectral  density  is  not  known,  but  subject  to  linear 

constraint  of  the  form 


^(a))a(c*))  dao  <  1  . 
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For  example,  if  the  mean  square  signal  acceleration  is  limited, 
we  would  have  a(co)  proportional  to  ;  for  limited  mean  square 
signal  velocity,  a(co)  is  proportional  to  'sr . 

By  use  of  Lagrange  multipliers  and  calculus  of  varia¬ 
tions  it  is  found  that  the  minimax  filter  satisfies 


1  -  F 


<f*,Q 


Xa(u)> 


<  Xo(cd) 


cd  :  >  0 

co  :  -  0, 


$  $  4* 

with  F^*  n  the  optimal  filter  for  S'  and  0,  <!“  the  maximum 

,  W 

signal  spectral  density,  and  X  chosen  to  satisfy  the  constraint. 

At  this  point  it  may  appear  difficult  to  determine  the 
minimax  filter,  as  the  solution  given  is  far  from  explicit.  For 
simple  forms  of  noise  density  0(gd),  however,  the  solution  is 
easy.  For  white  noise,  O(cd)  constant,  one  can  show  that 


1  -  F. 


**,© 


it*  +  © 


4  2 

To  illustrate  the  procedure,  consider  a  =  co  /a  ,  indi¬ 
cating  limited  mean  square  target  acceleration^  Equating  our 
expressions  for  jl  -  F  +  q|  >  and  setting  a^/x  =  a)*  , 


Q 

$>*  +  © 


cd  :  <£•  *  (cd)  >  0 


co  :  $  *(cd)  =  0 
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giving 


w  is  determined  by  the  constraint 
o 


The  minimax  F  can  only  be  approximated  in  practice.  A  very 
good  approximation  is  similar  to  the  filter  against  accelerating 
targets,  mentioned  by  Hanson,  but  with  a  damping  factor  of  V3/8 
rather  than  -/1/2.  That  is, 


1  +  V3/2  p/to  Q 
1  +  V3/2  p/cj0o  +  p2/u>o2 


Estimating  I nput  Spectral  Densities - Perhaps  the  most 

obvious  method  for  coping  with  a  variety  of  input  spectral  den¬ 
sities  is  to  process  the  input  to  estimate  the  spectral  densi¬ 
ties  and  choose  the  filter  on  the  basis  of  these  estimates. 
There  are  considerable  difficulties  associated  with  this 
approach,  beginning  with  the  problem  of  estimating  two  spectral 
densities  on  the  basis  of  one  time  series,  and  suitably  chang¬ 
ing  filter  characteristics  based  on  these  estimates.  With 
this  method  there  is  no  check  on  the  correctness  of  the  adjust¬ 
ment,  which  is  open  loop.  The  final  problem  is  that  in  radar 
tracking  and  missile  guidance  the  input  is  typically  not  avail¬ 
able  -  geometry  provides  feedback  and  only  the  so-called  error 
signal  is  measured. 


do), 
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Adjusting  Error  Spectral  Densities - A  key  to  closed 

loop  adjustment  of  the  filter  characteristics  is  the  relation , 
stated  above,  that  for  white  noise, 


1  -  F, 


0 

chpf  • 


Since  the  transfer  function  between  input  and  error 
X_  -  X 

signal  is  — ^ =  l  _  F(u>),  this  implies  that  if  the  filter- 
XI 

ing  is  optimal ,  the  spectral  density  of  the  error  signal  is 


(O  +  $>)  1  -  F. 


Closed  loop  adjustment  of  the  filter  is  thus  possible 
by  changing  filter  characteristics  in  such  a  way  as  to  main¬ 
tain  the  error  signal  spectrum  approximately  flat.  In  prac¬ 
tical  applications  of  this  concept,  the  error  signal  energy 
in  a  low  pass  band  has  been  compared  with  that  in  a  higher 

pass  band.  A  filter  of  standard  form  F  =  - *  +.  ^ £  ..T  P . ... 

1  +  2£  Tp  +  T^p 

has  been  used,  with  t  varied  at  a  rate  depending  on  the 
ratio  of  energies.  This  simple  scheme  has  resulted  in  ex¬ 
cellent  adaptation  to  varying  amounts  of  signal  and  noise,  re¬ 
sulting  in  near  optimal  filtering. 


Optimal  Filtering  in  Transient  and  Non-Stat ionary  Cases - 
By  exploiting  the  principles  of  least  squares,- it  is  possible 
in  many  cases  to  design  a  filter  which  minimizes  mean  square 
errors  in  the  transient  response  as  well  as  in  equilibrium, 
and  which  can  cope  with  nonstationary  inputs.  This  extension 
from  filtering  which  is  optimal  only  in  steady  state  and  re¬ 
quires  stationary  signal  and  noise  certainly  increases  the 
adaptivity. 

One  would  like  to  apply  to  this  type  of  filtering  some 
of  the  adaptive  methods  mentioned  above  -  minimax  principles, 
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estimation  of  input  spectra,  and  closed  loop  adjustment  based 
on  the  spectrum  of  error  signals.  Only  the  first  two  of  these 
principles  have  been  applied.  Minimax  design  is  considerably 
more  difficult  than  in  the  case  of  the  Wiener  optimal  filter, 
but  has  been  accomplished  by  Busey  in  at  least  one  problem. 

The  basic  method  of  adaptation  is  estimation,  by  sampling,  of 
the  spectra  of  the  noise  and  of  signal  driving  functions;  in 
some  applications  the  raw  material  has  been  available.  Sim¬ 
plified  closed  loop  filter  adjustment  based  on  the  error  sig¬ 
nal  spectrum  does  not  appear  possible.  One  could  use  meas¬ 
ured  deviations  of  the  error  signal  spectral  density  to  ad¬ 
just  estimates  of  strength  of  noise  and  signal  driving  com¬ 
ponents,  thus  using  closed  loop  adjustment,  but  it  is  by  no 
means  clear  that  this  would  be  superior  to  direct  estimation. 

Limiting  RMS  Output  Acceleration - The  RMS  output 

acceleration  of  a  missile  must  be  limited.  A  system  designed 
by  Follin  to  adapt  the  filtering  to  limit  RMS  acceleration  due 
to  noise  and  avoid  the  most  harmful  effects  of  hitting  accel¬ 
eration  limits  is  described  in  Reference  (1),  pp.  20-21. 


IV.  COMPENSATING  FOR  UNKNOWN  TRANSFER  FUNCTIONS 


It  is  frequently  the  case,  particularly  in  autopilot 
design,  that  a  variable,  not  completely  known  transfer  function 
is  a  significant  part  of  the  servo  loop.  In  autopilot  design, 
one  may  attempt  to  compensate  by  varying  gains  as  functions  of 
ram  or  static  pressure,  but  there  will  often  be  large  residual 
variations  not  properly  compensated.  The  bang-bang  systems 
mentioned  earlier  have  often  been  suggested  for  autopilots  to 
cope  with  these  variations.  In  practice,  however,  the  auto¬ 
pilot  is  typically  compensated  as  well  as  feasible  for  pres¬ 
sure  variations,  and  parameters  selected  to  yield  compromise 
performance  over  the  region  of  possible  variation.  We  shall 
now  consider  some  special  techniques  for  cases  in  which  such 
compromise  is  inadequate . 


Maximizing  Stable  Regulator  Gain - The  first  problem 

considered  is  to  maximize  regulator  gain  without  instability. 
This  problem  arose  in  attempting  to  control  roll  precisely  in 
a  missile  with  enormous  variation  in  aerodynamic  gain.  With 
standard  techniques,  the  roll  autopilot  gain  had  to  be  kept 
low  to  avoid  instability  with  high  aerodynamic  gain;  roll 
control  was  inadequate  with  low  aerodynamic  gain.  The  solu¬ 
tion  depended  on  the  fact  that  the  resonant  frequency  was 
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nearly  unstable,  the  aerodynamic  changes  being  primarily  in 
gain.  Energy  in  this  frequency  region  was  detected,  and  com¬ 
pared  with  energy  calculated  for  neutral  stability.  The 
difference  was  used  to  vary  the  autopilot  gain.  The  difference 
in  performance  of  the  system,  according  to  whether  the  energy 
is  measured  before  or  after  the  variable  gain,  is  significant. 
Both  can  be  made  stable  and  are  correct  in  steady  state,  but 
the  closed  loop  system  has  zero  velocity  lag  and  transient 
response  independent  of  the  critical  gain. 

Sensitivity  Feedback - One  of  the  least  desirable  types 

of  autopilot,  from  the  theoretical  standpoint,  is  a  wing  posi¬ 
tion  control  system,  which  attempts  to  make  wing  deflection 
proportional  to  demanded  acceleration.  Such  autopilots  have 
been  used,  however,  either  because  of  their  great  simplicity  or 
because  the  alternative  accelerometer  feedback  system  causes 
trouble  due  to  amplification  of  a  missile  body  vibration  and 
bending.  Satisfactory  operation  of  a  wing  position  control 
system  requires  adequate  knowledge  of  the  ratio  of  acceleration 
to  wing  position.  A  solution  proposed  by  T.  W.  Sheppard  was 
to  compare  average  acceleration  with  average  demanded  accelera¬ 
tion,  and  use  this  comparison  to  adjust  the  gain  of  the  auto¬ 
pilot.  This  approach  was  found  to  be  feasible  even  in  some 
quite  complicated  cases.  It  was  also  found,  the  hard  way, 
that  sufficiently  shoddy  components  for  changing  the  gain  could 
invalidate  any  performance  analysis.  There  is  still  a  require¬ 
ment  for  compact,  reliable,  reasonably  precise  gain-changing 
devices  for  use  in  guided  missiles. 

Tracer  Signals - In  the  laboratory,  a  common  method  for 

determining  system  response  is  to  insert  signals  at  the  input 
and  observe  the  output.  The  use  of  this  method  in  flight 
with  autopilots  could  furnish  useful  information  on  airfx*ame 
and  other  responses.  Tracer  signals  of  various  sorts  have 
been  proposed  for  various  purposes,  but  are  usually  rejected 
because  of  interference  with  flight  or  strong  dependence  on 
details  of  dead  space,  friction,  etc.,  when  too  small  or  too 
high  frequency  to  disrupt  normal  missile  operation. 

Tracer  signals  can  be  used  effectively  in  many  systems 
which  function  intermittently.  As  an  example,  they  can  facili¬ 
tate  adjustment  of  gains  of  pulse  radar  IF  strips. 
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ABSTRACT 


Classical  optimal  filtering  methods  have  been  ex¬ 
tended  to  a  large  class  of  problems  in  which  the  input  has 
incompletely  specified  characteristics.  By  minimax  princi¬ 
ples  the  optimal  filter  and  the  best  input  are  determined. 
Two  problems  of  time-varying  filters  are  considered,  first 
the  optimal  settling  of  filters  to  steady  state  and  second 
the  design  of  adaptive  filters  which  adjust  to  varying  or 
unknown  environment. 
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I  INTRODUCTION 


Linear  filtering  theory  is  largely  based  on  the 
fundamental  work  of  Wiener,  "Extrapolation,  Interpolation, 
and  Smoothing  of  Stationary  Time  Series,"  1950,  which  in 
many  respects  paralleled  the  independent  work  of  Kolmogorov. 

It  may  be  well  to  review  the  problem  considered  by 
Wiener.  He  assumed  that  there  was  available  the  entire  past 
history  of  a  time  series  consisting  of  signal  plus  noise  pos¬ 
sibly  correlated  with  the  signal,  that  all  processes  involved 
were  stationary  and  indeed  ergodic  to  the  second  order,  with 
known  auto-correlation  and  cross-correlation  functions.  He 
wished  to  determine  the  realizable  linear  filter  to  apply  to 
the  signal  in  order  to  minimize  the  mean  square  difference  be¬ 
tween  the  output  and  the  message  translated  by  an  assigned 
positive  or  negative  time  interval.  Wiener  solved  this  prob¬ 
lem  by  using  variational  analysis  on  the  weighting  function  to 
obtain  an  integral  equation,  then  by  using  subtle  Fourier  anal¬ 
ysis  to  solve  the  integral  equation.  Multiple  time  series  were 
handled  by  an  extension  of  this  technique. 

It  will  be  noted  that  the  problem  solved  by  Wiener 
contains  two  restrictions  beyond  the  assumptions:  first,  op¬ 
timization  is  restricted  to  linear  functions;  second,  the  loss 
function  whose  expectation  is  minimized  is  the  squared  error. 

In  attempting  to  extend  filtering  theory,  it  is 
appropriate  to  modify  or  eliminate  various  of  these  assumptions 
or  restrictions.  Various  investigators  have  widened  the  field 
of  permissible  filters  and  dealt  with  alternative  loss  functions. 
Nonstationary  processes  have  been  considered;  some  trivial  ob¬ 
vious  results  have  been  obtained,  and  some  adaptive  filters 
appear  suitable,  but  little  has  been  done  that  is  both  signifi¬ 
cant  and  rigorous.  In  this  paper  we  shall  remove  the  assump¬ 
tion  of  complete  knowledge  of  the  correlation  functions  and 
also  indicate  some  minor  extensions  of  the  basic  theory  and 
techniques,  consider  the  optimum  filter  with  only  portions  of 
the  signal  history  available  and  attempt  to  classify  the  types 
of  adaptive  filters. 


II.  THE  FREQUENCY  APPROACH 


The  reader  of  Wiener's  work  will  note  that  although 
his  basic  problem  is  formulated  in  terms  of  time  series, 
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correlation  functions,  and  weighting  functions,  his  solutions 
are  expressed  in  terms  of  spectral  densities  and  transfer 
functions.  It  appears  reasonable,  consequently,  to  set  up 
the  problem  in  the  frequency  domain.  From  the  elementary 
properties  of  spectral  densities,  we  have 


(2.1) 


den  . 


2 

where  a  is  the  mean  square  error,  F(o>)  the  filter  transfer 
function,  a  the  time  translation,  and  M  and  N  the  signal 
noise  spectral  densities,  so  normalized  that  the  signal  power 
is  j  M(cn)  do).  All  integrals  are  taken  over  the  entire  real 
frequency  axis  unless  otherwise  indicated,  and  the  dependence 
of  the  variables  on  cn  will  usually  not  be  indicated.  It  is 
assumed  here  and  henceforth  that  the  signal  and  noise  are  in¬ 
dependent;  this  entails  no  loss  in  generality  in  the  classi¬ 
cal  developments,  where  M  can  be  regarded  as  the  sum  of  the 
signal  and  signal  on  noise  spectral  densities,  N  as  the  sum 
of  the  noise  and  noise  on  signal  spectral  densities. 

It  is  useful  to  consider  the  spectral  densities  as 
resolutions  of  the  signal  power  into  a  continuum  of  frequency 
components.  From  this  standpoint  it  is  clear  that  ergodic 
properties  are  not  relevant  to  the  problem  of  linear  filtering, 
although  the  optimal  filter  may  be  nonlinear  if  the  second- 
order  characteristics  are  not  ergodic. 

2 

Before  applying  variations  to  minimize  a  by  choice 
of  F,  let  us  indicate  some  extensions  of  this  relation  to 
problems  beyond  the  original  one.  In  the  first  place  it  will 
be  noted  that  Eq.  (2.1)  is  valid  even  though  the  power  of  sig¬ 
nal,  noise,  or  both  is  unbounded;  it  is  not  necessary  that  the 
correlation  function  exist.  It  will  be  seen  that  this  may  be 
of  importance. 

A  trivial  generalization  of  Eq.  (2.1)  is  to  replace 

eia)a  by  the  Fourier  transform  Y(o>)  of  any  desired  linear 
operator  on  the  signal,  e.g.  by  iau  for  the  derivative.  Thus, 


(2.2) 


2 

o 


do). 
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Another  easy  extension  is  to  the  case  in  which  it 
is  desired  to  weight  errors  unequally  for  various  frequency 
components.  A  symmetric  non-negative  function  W(co),  so 
normalized  that  f  W(o>)  da>  -  1 ,  could  be  inserted  to  obtain 


(2.3) 


W  dx  , 


which  is  of  course  formally  equivalent  to  Eq.  (2.2)  with  M 
and  N  replaced  by  MW  and  NW. 

2 

The  next  extension  is  to  minimize  o  subject  to  a 
restriction  on  the  mean  power  of  the  output  or  some  other  linear 
function  of  the  output.  As  an  example,  if  must  be  minimized 
subject  to  the  restriction  that  the  output  acceleration  power 
must  be  less  than  fl,  i.e.,  that 


(2.4) 


f(M  +  N) 


dec  /3 , 


we  should  minimize 


(2.5) 


+ 


^  lY 


N  +  A  (M  +  N) 


den, 


and  select  A.  to  satisfy  Eq .  (2.4).  In  certain  cases, 
especially  with  non-gaussian  processes,  restraints  such  as 
Eq.  (2.4)  may  preferably  be  applied  only  to  the  noise.  In 
this  case  the  integral  to  be  minimized  can  be  reduced  to  Eq. 
(2.2)  by  suitable  definition  of  N.  This  type  of  side  condi¬ 
tion  can  be  introduced  formally  as  here,  or  can  be  used  in 
the  definition  of  the  class  over  which  F  is  optimized. 
Several  simultaneous  side  conditions  can  be  introduced  in  the 
same  way. 


The  final  extension  to  be  considered  is  to  replace 
the  class  of  relizable  transfer  functions  by  other  classes, 
say  ,  as  appropriate. 
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III . 


MINIMIZATION  OF  o2 


BY  F 


We  now  consider  the  selection  of  F  within  the 

class  to  minimize  o2 .  An  increment  J  or  F  will  produce 

2 

an  incremental  a  of 


(3.1) 


+  F  -  o 


■/I 


jr(M  +  N)  da)  + 


/j  [< 


(M  +  N)  F  -  MY 


D 


a  relation  obtained  by  noting  that  every  transfer  function  has 
an  even  real  part  and  an  odd  imaginary  part. 

The  transfer  function  F  will  be  optimal  in  5^  if 
the  right-hand  integral  of  Eq.  (3.1)  is  non-negative  for  every 
J  such  that  J  +  FC^pand  ^|j|^  (M  +  N)  dcu  is  finite. 

The  absolutely  optimal  F  is,  from  Eq.  (3.1),  evi¬ 
dently 


MY 

(3-2>  Fo  '  rm  • 


The  optimal  realizable  F  is  given  by 


MY 

(M  +  N)~ 


where  the  new  symbols  denote  factorization  and  decomposition 
of  a  meromorphic  function  as 


(3.3) 


(M  +  N)  + 


(3.4)  H  C-  H+  H"  +  H+  +  H 


dto, 
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H+  being  analytic  and  without  poles  or  zeroes  in  the  lower  half 
plane,  H“  the  conjugate  of  H+;  H+  and  H_  have  no  poles  in  the 
lower  half  plane  and  upper  half  plane,  respectively.  Polynomial 
terms  of  H  appear  in  H+.  If  It  +  N  is  not  factorable,  FR  ■  FQ . 

To  check  the  validity  of  Eq.  (3.3),  substitute  it  in 
Eq.  (3.1),  obtaining 


cv  2  2 

(3'5)  °J  +  FR  "  °Fr 


J |  j| 2  (M  +  N)  do 


+  f  1  (M  +  N) 


MY 


(M  -I-  N) 


do; 


the  latter  integral  is  zero,  by  contour  integration  over  the 
upper  half  plane,  for  J  sufficiently  convergent,  i.e.,  for 

/lJ|2  01  +  N)  do  finite . 

As  an  example  let  us  consider  the  spectra 


(3.6)  M  -  O/to4  ,  N  -  4  ,  with  Y  -  1 


the  absolute  optimum  (realizable  with  infinite  delay)  is  then 


(3.7) 


F0<P) 


i  +  o  p’/e 


and  the  optimal  realizable  F  is 
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(3.8) 


FR(P)  - 


1  4-  42  P /^/0)V4 
1  +/2p  (**/ 0)1/4  +  p2  (rf/©)1/2 


This  transfer  function  is  the  zero  velocity  lag  loop  with  0.7 
critical  damping  which  will  be  discussed  below.  The  corre¬ 
sponding  errors  are 


(3.9) 


o2  -  .3535  d3/4  01/4 


2  .  ...  ,3/4  q1/4 

Op  “  1.414  o  © 


A  very  useful  relation  can  be  obtained  by  manipula¬ 
tion  of  the  second  integral  of  Eq.  (3.1)  as  follows: 


(3.10)  J  J  [  (M  +  N)  F  -  1IY]  du>  -  Jl/T  [  (M  +  N)  |  F  |2  -  MYF] 
-  J  J/F  l  (M  +  N)  f|2  -  ReMYF  -  Im  MYF)  du>  . 


This  integral  is  zero  for  sufficiently  convergent 
realizable  J  if 


duo 


(3.11)  F, 


(M  +  N)  =  Re  MYF  +  Re : Im  MYF  , 


where  Re:ImH  represents  the  real  complement  to  the  given 
imaginary  function  such  as  to  render  H  realizable. 

This  can  be  more  formally  expressed  by  the  Bode 

relation 
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(3.12) 


(M  4  N) 


w  (MYF)w  dw 
_  w 


where 


indi¬ 


cates  disregard  of  poles  at  w  *  ±  d,  provided  MY  docs  not  di 
verge  for  large  oo. 

Solutions  of  Eq.  (3.11)  for  simple  forms  of  MY  are 
readily  obtained;  for  example, 


(3.13) 


Fr  *  (M  +  N) 


r 


MY,  if  MY  is  constant 


J  MYf  (ia)  ,  if  MY  e  -£-^- 


V. 


a  4-  jd 


MY  +  k,  if  MY  »  cq  +  c2  a)2  +  c 4  jo4 


Q 


k  determined 


by  J 


log  |  FR|  2  d  -  6]  . 


By  symmetry  under  the  interchange  of  M<-*  N  and  F<-^ 
(Y  -  F)  we  obtain 


(3.14)  |y  -  FR  2  (M  4  N)  -  NY,  if  NY  is  constant. 


For  ordinary  filtering,  with  Y  -  1 ,  these  define  the  optimal 
spectral  density  of  the  error  signal  in  a  servo  type  filter, 
and  can  be  used  to  construct  an  adaptive  filter  which  will  be 
come  optimal  for  any  signal  spectrum. 

I  1 2 

Expressions  for  FR  ,  based  on  Eq.  (3.71)  are 

quite  useful  in  optimizing  filters  with  side  conditions  such 

as  limited  mean  square  output,  since  the  expressions  for  |fr| 

are  frequently  more  convenient  to  use  than  those  for  FR  in 
determining  the  Lagrange  multipliers. 
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IV.  MINIMAX  FILTERING 


Let  us  consider  now  some  typically  statistical  prob¬ 
lems,  in  which  one  has  incomplete  knowledge  of  the  spectrum 
of  noise,  of  signal,  or  both.  We  discuss  below  adaptive  quasi- 
linear  filters  which  appear  suitable  for  cases  in  which  one 
ectrura  is  completely  unknown,  and  which  can  cope  with  cases 
volving  a  spectral  density  of  known  form  but  unknown  magni- 
ude. 


The  problems  considered  by  Wiener  were  essentially 
probabilistic ,  i.e.,  the  system  is  completely  described  in 
terms  of  appropriate  probability  measures;  the  problems  we  are 
now  considering  are  statistical ,  in  that  we  are  dealing  with  a 
system  defined  by  probabilities,  some  of  which  are  unknown. 

Our  problem  in  this  case  is  one  of  statistical  estimation  of 
a  function  of  the  signal.  Our  estimating  function  should  be 
optimal  in  some  sense.  One  of  the  most  logical  criteria  for 
an  estimate,  developed  by  Abraham  Wald,  is  that  it  minimizes 
the  maximum  expected  loss;  that  is,  each  filter  is  assessed 
on  the  basis  of  the  expected  loss  with  the  possible  system 
which  is  least  favorable  for  the  given  filter,  and  the  optimal 
filter  is  that  filter  for  which  the  maximum  expected  loss  is 
minimum.  This  formulation  of  statistical  decision  theory  is 
very  similar  to  two-person  game  theory,  independently  de¬ 
veloped  by  von  Neuman.  We  adopt  this  criterion  and  consider 
as  optimal  the  minimax  filter,  with  the  loss  function  propor¬ 
tional  to  the  squared  error.  In  a  completely  prescribed  sys¬ 
tem,  the  minimax  linear  filter  is  the  Wiener  optimal  linear 
filter . 


Minimax  theory  offers  a  strong  justification  for  the 
use  of  linear  filters.  If  the  distribution  functions  of  the 
processes  are  not  kr.own  but  the  class  of  possible  distributions 
includes  Gaussian  distributions,  the  minimax  filter  is  linear, 
since  with  a  linear  filter  the  mean  square  error  is  independent 
of  the  form  of  the  distribution  function,  and  with  a  non-linear 
filter  the  mean  square  error  exceeds  that  with  a  linear  filter 
when  the  processes  are  Gaussian. 

Typical  problems  encountered  in  practice  involve 
situations  in  which  the  noise  process  is  known  to  be  limited  in 
power,  in  mean  square  velocity,  etc.: 


(4.1)  /  N  d^>  =  C(  J'  n  -d2  =  Cj ,  etc  . 
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Such  restrictions  can  be  put  in  the  general  form 


(4.2) 


/ 


N  0  d  'JD 


1, 


where  0  is  a  prescribed  symmetric  non-negative  function,  K  a 
prescribed  positive  number.  It  can  be  shown  straightforwardly 
that  the  variation  in  due  to  variation  n  in  N  is  a  non¬ 
negative  function  of  n,  zero  if  and  only  if  n  -  0,  plus 


(4.3) 


where  Fu  „  is  the  optimal  transfer  function  with  M  and  N, 
and  A  is ’a  Lagrange  multiplier  to  be  selected  to  satisfy  Eq. 
(4.2).  From  EQ .  (4.3)  and  the  fact  that  n  ♦  N  must  be  non¬ 
negative,  it  follows  that  the  maximum  N  is  Nq  given  by 


(4.4)  |fMjN  |  2  -AO 


jj:  N  >  0 

o 


<  A  © 


i): 


N 


o 


-  0. 


This  result  is  easily  generalized  to  the  case  of 
several  inequality  restrictions 


(4.5)  Kj  J  NOj  djo  -  1  (j  -  1,2...,  k) 

with  the  K  not  specified  but  >  1. 

The  maximin  N  is  Nq  satisfying 
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(4.G) 


2  ‘  S  xj  kj  cj 
1  2  ‘j  kj  8j 


jo:  N  >  0 

o 

jj:  N  -  0 

o 


with  the  ,  K.  satisfying  Eq .  (4.5)  and  also 

(4.7)  Aj  (Kj  -  1)  -  0.  (j  -  1,2. . . ,  k)  . 


Similar  results  are  obtained  for  cases  in  which  the 
signal  spectral  density  is  subject  to  one  or  more  equalities 
or  inequalities,  and  where  both  spectra  are  limited  only  by 
such  restrictions. 

The  results  just  given  have  derived  the  maximin 
spectrum  or  spectra,  but  our  object  is  to  determine  the  mini¬ 
max  filter.  For  this  purpose  we  now  prove  that 


Min  Max 
F  N 


Max 

N 


ant  that  the  minimax 
that 


is  F, 


M,N 


To  prove  this,  we  observe 


2  2 

Min  Max  oN  „  <,  Max  o.,  „ 

F  N  *’*  W,*M.N 


M: 


[n  fs,  m  1  2  *  « 

Y  -  F 

M,N 

1 1 

«l 

o 

2  . 
Gi3 


+  J M  |y 


Max  Min 
N  F 


2 

°N.F  ' 
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Hi?  have  thus  shown  that 


(4.8)  llin  Max  at,  F  <  Mr  :  Min 

F  N  N,lf  N  F 


N .  F 


But  by  the  fundamental  theorem  of  game  theory, 


(4.9)  Min  Max  „  >  Max  Min  oi?  _  . 

F  N  N  F 


from  which  it  follows  at  once  that  the  "game"  is  determined, 
with  the  minimax  filter  being  Fu  „  and  the  maximin  N  being 

’  o 

the  Nq  previously  defined. 

The  basic  result  of  the  minimax  approach  to  optimum 
filtering  is  that  the  errors  depend  in  the  second  order  cn  the 
spectra  and  the  form  of  the  filter.  As  a  consequence  if  a 
suitable  approximation  to  the  optimal  form  is  used  and  the 
parameters  are  adjusted  properly  the  resulting  system  will  be 
satisfactory. 


V.  TIME  VARYING  FILTERS 


Let  us  now  consider  the  problem  of  filtering  when 
only  a  finite  and  perhaps  fragmentary  history  of  the  signals 
is  available.  In  this  case  the  filter  parameters  are  varia¬ 
ble,  and  we  must  assume  a  particular  form  of  transfer  func¬ 
tion.  In  general  the  steady  state  optimal  filter  with  varia¬ 
ble  band-pass  and  damping  is  best.  We  may  attack  this  prob¬ 
lem  in  the  time  domain  by  considering  the  rate  of  change  of 
the  filtering  errors  and  adjusting  parameters  to  maximize 
the  rate  of  decrease  of  the  error. 

For  a  linear  system  the  tracking  accuracy  may  be 
described  in  terms  of  the  variances  of  the  tracking  error 
and  error  rate.  As  an  example  let  us  consider  the  simple 
zero  velocity  lag  feedback  system  in  Fig.  1.  The  input 
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consisting  of  signal  and  noise  x(t)  +  x  (t)  is  at  the  left, 

n 


the 

output 

,  x  , 

c 

at 

the 

right.  The  equations  of  the  system 

are 

dx 

(5  . 

1) 

c 

dt 

X 

c 

+  b 

(x  -  x  )  +  b  x 
c  n 

dx  • 

c 

dt 

a 

(x  - 

x  )  +  a  x 
c  n 

Note,  in  explanation  of  the  notation,  that  x^  ^  x^  . 

Actually  the  principal  interest  centers  upon  the 

errors  e  =  x-  x,€  =x-x.  .  In  the  second  diagram  is 

c  c  c  c 

shown  the  error  loop  equivalent  to  the  original  signal  loop. 
The  signal  x  now  appears  as  an  acceleration  input  to  the 
first  integrator.  This  is  a  significant  advantage  when,  as 
is  often  the  case,  the  acceleration  spectrum  of  the  signal  is 
known  . 


So  far  a,  b  are  unrestricted.  If  x  and  xn  both  have 

flat  spectral  densities  9,  respectively  then  this  filter  is 
optimal  with  the  values  of  a  and  b  previously  derived. 

The  present  purpose  is  to  extend  the  optimization  to  the 
transient  period.  The  gains  a,  b  in  this  case  are  time 
functions  and  the  resulting  system,  while  not  necessarily 
optimum  among  all  possible  systems,  is  the  best  obtainable 
with  a  given  structure.  The  key  to  the  solution  lies  in  setting 
up  the  differential  equations  relating  the  variances  and  co- 
variances  of  the  integrator  outputs  t  .  .  Write  Eq .  (5.1) 

in  terms  of  the  errors  and  express  the  solution  in  the  neighbor¬ 
hood  of  t  as  a  power  series  in  At.  Terms  beyond  the  first 
degree  in  \t  are  not  required.  The  result  is  most  easily  ob¬ 
tained  by  direct  use  of  the  second  figure 

t+  t 

(5.2)  ec  =  cc  +  cc  ^  *  b  €c  ^  b  I  xn  dt  +  0  (it2) 
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(5.3) 


€  •  -  a  £ 
c  ^  c 

o  o 


t+At 

+  f  x  dt  +  a  J 

t 


t+At 

x 


n 


dt  +  O  (At  ) 


Square  Eq .  (5.2)  and  average  over  the  ensemble  of 
inputs  x,  x^ .  Then  denoting  resulting  variances  and  covari¬ 
ances  by ‘T  'e  •  'e'  •  respectively 

c  c  cc 


(5.4) 


where  0  is  the  spectral  density  (assumed  constant  of  x^). 

The  term  d  is  derived  as  follows:  el  (a)  is  the  autocorrelation 

function  of  x  . 

n 


t+At 


dt 


t+At 

/  xn  dt> 
t 


t+At 


/ 


(u)  xn  (v)^>  du  dv 


At  At 


I  I 


d  (u  -  v)  du  dv 


=  el 


At 


/ 


At 


I 


6 (u  -  v)  du  dv 


=  el  At  . 
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Dividing  Eq .  (5.4)  by  and  letting 


d£  .  . 

(5.5)  -gf  -  2  tc-  -  2  bfc  +  b2  * 


is  flat 


Similarly,  assuming  the  spectral  density  ©  of  x 


dfc  /v  2 

(5.6)  -gf  =  -2  a  ?c  •  +  a 2  d  -  © 


(5.7) 


CC  A 


— rr —  -  €  •  —  a  £  -  b  £  ■  +  a  b  p5 

dt  c  c  cc 


These  variance  and  covariance  equations  may  be  used 
to  adjust  the  gains  to  get  optimum  tracking.  This  will  surely 
result  if  functions  a,  b  can  be  found  making  the  right  hand 
members  of  Eqs .  (5.5),  (5.6),  and  (5.7)  simultaneously  minimum 

for  this  will  make  /t  and  decrease  at  maximum  rate.  This 

c  c 

simultaneous  minimum  does  occur  and  at 


(5.8) 


as  can  be  seen  by  setting  the  partial  derivatives  of  all  three 
right  Members  with  respect  to  a  and  b  equal  to  zero.  The 
resuming  system  has  optimum  tracking  and  rate  of  settling  and 
the  variances  facilitate  evaluating  performance  of  the  system. 

This  optimization  of  the  transient  behavior  has  a  by 
product-the  known  steady-state  result.  For  in  this  case  the 
left  hand  members  of  Eqs.  (5.5),  (5.6),  and  (5.7)  vanish  when 
us ing  Eq .  (5.8) 
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1/4 

(steady-state) 


For  simplicity  the  input  acceleration  spectrum  has 
been  assumed  flat.  There  are  several  ways  to  deal  with  non¬ 
flat  spectra.  For  example  it  can  be  shown  that  for  a  general 
0  (oj)  with  autocorrelation  function  b(t),  ©  in  Eq.  (5.6)  would 

t 

be  replaced  by  2 J  W  (t,  t)  d  (t  -  t)  dx.  Here  W  (t,  t)  is  the 

o 

impulsive  response  of  ^  to  x.  Or  a  flat  spectrum  could  be 

be  filtereu  by  ©  (m)  4l©(a>)  =  0(x)  +  0(:jo)”)  to  give  a  signal 
of  spectrum  ©(a))  into  the  integrator.  This  last  is  particu- 

Q  1 

larly  simple  in  the  Markoff  case  0  -  —n — ~ i*  whore  ©+  *  - 

+  aT  C  +  jo; 

The  original  error  tracking  loop  could  be  modified  as  shown  in 
the  third  figure  thus  introducing  one  additional  integrator. 
Proceeding  as  before  six  variance  -  covariance  equations  re-., 
suit.  In  general  with  a  system  involving  n  integrators  (n1 

order  differential  operation)  there  will  be  — ^ - -  variance 

equations  although  generally  some  are  trivial. 

The  transient  filtering  problem  has  been  discussed  on 
the  basis  that  the  noise  and  signal  spectra  are  known.  This 
leads  to  a  solution  of  the  optimum  settling  time  of  a  filter 
and  to  the  best  combination  of  parameters  even  if  the  transfer 
function  is  not  optimal.  If  the  noise  and  signal  spectra  are 
not  known  then  the  above  technique  of  computing  variances 
fails  and  other  methods  must  be  used.  If  sufficient  time  is 
available  it  is  possible  to  measure  the  spectral  densities 
and  use  the  variance  methods  but  less  cumbersome  methods  are 
desirable . 


(5.9) 


a  "(§) 


1/2 


b  =  y[2 


/e\ 


VI.  ADAPTIVE  FILTERS 


The  discussion  which  follows  will  concern  adaptive 
systems,  i.e.,  those  which  change  parameters  or  adapt  as  a 
function  of  the  environment.  In  general  the  rate  of  change 
of  parameters  is  slow  compared  to  the  data  rate  of  the  input 
so  that  they  may  be  treated  as  time  varying  rather  than 
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non-linear  systems.  If  the  response  of  the  adapting  loop  is 
fast,  paper  analysis  is  impossible  and  simulation  techniques 
are  needed.  In  designing  an  adaptive  system  it  is  necessary 
to  consider  the  response  and  stability  of  the  loop  as  well 
as  the  source  of  intelligence  to  be  employed  in  adjusting 
the  system. 

Adaptive  systems  may  be  classified  according  to 
several  distinct  criteria  as  follows: 

1  Object  of  adaptive  loop 

a.  Setting  of  gain  or  transfer  function 
b  Adjusting  output  level  or  other  parameter 
c.  Adjusting  stability  margins  of  main  loop 

2.  Source  of  information 

a.  Measurements  of  normal  input  or  output 

b.  Injection  of  tracer  signal  outside  band 
pass  of  normal  input 

c.  Time  sharing  tracer  signal 

d.  Amplitude  or  phase  of  self  excited  oscilla¬ 
tions 

3.  Type  of  system 

a.  Open  loop,  i.e  ,  system  adjusted  according 
to  measurements  on  the  input 

b.  Closed  loop,  i.e.,  system  measures  signal 
or  tracer  output. 

In  addition  all  adaptive  systems  may  be  classified 
according  to  standard  servo  practice  as  electrical  or  mechani¬ 
cal,  digital  or  analogue,  etc.,  but  such  distinctions  are  not 
desirable  for  the  present  purpose.  The  class  of  adaptive 
servos  ranges  from  standard  AGC  and  AFC  loops  to  servo  driven 
autotransformers  for  voltage  regulation  to  more  sophisticated 
optimal  filtering  loops. 

Let  us  now  look  at  examples  of  the  three  different 
adaptive  systems  listed  under  criterion  1.  These  are  the 
zero  velocity  lag  tracking  loop  mentioned  earlier,  a  similar 
filter  with  limited  output  acceleration,  and  a  system  for 
maintaining  a  servo  loop  as  tight  as  possible  without  in¬ 
stability  when  the  loop  gain  is  slowly  varying  or  not  known 
accurately . 
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Figure  2  shows  a  simple  servo  where  only  the  gain 
is  varied  and  the  loop  gain  is  to  be  maximized  (possibly  to 
minimize  the  effects  of  a  variable  back  torque  from  the  load). 
The  unknown  gain  is  assumed  to  be  in  the  servo.  The  band- 
pass  filter  passes  the  frequency  at  which  instability  is  ex¬ 
pected  and  this  signal  is  detected  and  used  to  adjust  the 
gain  to  damp  the  oscillations.  Let  XQ  be  the  gain  at  which 

the  loop  is  neutrally  stable.  Then  the  rate  of  buildup  or 
decay  of  oscillations  is  proportioned  to  (X/X  -  1).  The 
amplitude,  z,  of  the  oscillations  satisfies  tne  equation 


(6.1)  z  -  kj  z(X  -  Xo)/Xq  -  k^  zQ  (X  -  Aq)/Xo 


If  the  signal  is  picked  off  at  point  A,  the  control  equation 
is 


(6.2)  X  -  -  g  (s)  (z  -  zQ) , 


while  if  it  is  picked  off  at  point  B,  we  have 


(6.3)  X  -  g  (s)  X  (z  -  zQ) 


-  g  (s)  (z  -  zo) 


Combining  these  equations  we  find  that  in  case  A 


(6.4) 


's2  ♦ 


k,  z 


:  Z1  1 

V*  (s) 

O  —I 


<s)  (*;) 
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while  for  case  B 


X  =  kj  Zj  g  (s)  XQ  . 


can  be  made  stable,  and  both  have 
X  -  Xq  as  the  steady  state  solution,  the  transient  response  of 

the  adaptive  loop  depends  on  the  required  gain  in  case  A, 
contrarily,  in  case  B  the  transient  is  fixed  and  the  system 
has  zero  velocity  lag  with  respect  to  variations  of  Xq.  If  XQ 

increases  linearly  with  time  then  X  =  X  but  z  >  z,  with  z  *=  0. 

o  1 

Thus  a  closed  loop  adaptive  system  is  better  here.  These  state¬ 
ments  are  subject  to  modification  if  there  exists  dead  space  or 
friction  it.  the  main  or  adaptive  loop;  actually  simulation  is 
then  required  to  determine  the  behavior. 

If  the  main  loop  shaping  network  f(s)  is  properly 
chosen  then  a  very  good  servo  response  is  obtainable  but  in¬ 
put  signals  at  the  loop  resonance  frequency  must  be  avoided. 

This  design  is  especially  useful  for  a  regulator,  i.e.,  when 
-  0  and  the  servo  is  designed  to  counter  the  back  torque. 

Figure  3  is  a  diagram  of  an  adaptive  filter  in  which 
the  RMS  output  acceleration  due  to  noise  is  limited.  The 
adaptive  loop  is  very  simple  and,  with  the  gains  as  shown, 
has  a  response  which  is  independent  of  input  noise  spectral 
density.  This  can  be  seen  from  the  fact  that  the  error  in  RMS 
acceleration  is  proportional  to  ~.X/X  and  hence  the  control 
equat ion  is 


(6.6)  X  ~  a. 


(6.5)  is  +  k 


l  Zj  B  (s) 


While  both  systems 


if  the  filter  g(s)  is  unity.  If  it  is  desired  to  have  the 
adaptive  loop  time  constant  proportional  to  the  main  loop  time 
constant  then  the  X  must  be  replaced  by  X2  and  the  control 
equation  is 


(6  .  /  )  X  —  X  X  . 
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If  the  noise  is  flat  or  of  known  spectral  shape  it 
is  possible  to  measure  its  amplitude  at  the  input  to  the 
filter  and  compute  the  RMS  acceleration  from  the  known  transfer 
function  of  the  filter.  It  is  then  possible  to  use  an  open 
loop  method  of  adjusting  X  but,  while  this  eliminates  adaptive 
loop  stability  problems  it  does  not  have  the  accuracy  in  adjust¬ 
ing  X.  If  the  noise  does  not  have  the  expected  spectrum  the 
filter  f(s)  will  not  be  optimal  in  shape,  however  the  performance 
of  the  system  will  deviate  in  the  second  order  if  X  is  correct, 
but  errors  in  X  may  give  first  order  effects  on  system  per¬ 
formance  due  to  violation  of  the  constraints. 

While  this  loop  is  very  simple  we  have  discussed  it 
because  some  simulator  studies  of  the  effects  of  non-linear ites 
may  be  of  interest.  The  non-linearity  concerned  is  that  due  to 
a  fixed  limiter  inserted  in  the  loop  as  shown  at  the  bottom  of 
Figure  3.  The  analysis  was  carried  out  using  various  fixed 
values  of  X  and  noise  rather  than  closing  the  adaptive  loop 
as  above.  Figure  4  shows  the  results  obtained.  The  solid  upper 
2 

curve  shows  a  vs  X  for  an  unlimited  system.  If  the  limiter 
is  inserted  then  the  dashed  curve  is  appropriate;  the  minimum 
is  only  a  few  percent  above  the  optimum  at  the  minimum.  The 
lower  half  of  the  figure  shows  the  effect  of  the  limiter  on 
the  output  acceleration  before  and  after  the  limit.  The  result 
of  the  simulator  study  was  that  the  minimum  in  occurs  almost 
exactly  at  a^  =  L,  hence  the  simple  adaptive  system  just  de¬ 
scribed  forms  a  very  sophisticated  tracking  loop. 

Figure  5  is  a  block  diagram  of  the  zero  velocity  lag 
tracking  loop  in  which  we  do  not  know  the  signal  or  noise 
spectra  although  we  assume  the  noise  to  he  nearly  flat.  In 
order  to  adjust  band-pass  we  may  use  the  result,  Eq .  (3.13)  ,  so 
that  the  error  signal  spectrum  is  proportional  to  the  noise 
spectrum  when  the  loop  is  optimal.  While  the  transfer  of  the 
loop  is  not  correct  if  the  noise  is  not  flat  or  if  the  signal 
is  not  that  assumed,  it  is  still  true  that  adjusting  the  loop 
band-pass  so  that  the  error  signal  spectrum  is  flat  is  nearly 
opt  inium 


The  method  of  measurement  is  to  use  two  filters 
f,(s),  a  low  pass  filter  covering  the  band-pass  of  the  main 
loop,  and  fQ(s)  covering  an  equal  band-pass  just  above  the 

main  loop  and  take  the  ratio  of  the  outputs.  The  optimum 
shape  of  such  filters  has  not  been  determined  but  simulator 
runs  show  suitable  performance  for  simple  filters.  fgCs) 

should  have  a  finite  band-pass  because  the  actual  high  fre¬ 
quency  noise  is  unimportant;  only  the  u  lse  in  the  vicinity 
of  the  main  loop  band-pass  is  important. 
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In  the  loop  as  shown  the  obvious  scale  factors  have 
been  inserted  to  make  the  response  frequency  -  relative  to 
the  main  loop  -  independent  of  the  value  of  the  spectra.  The 
filter  f^(s)  determines  the  band-pass  of  the  adaptive  loop  and, 


in  order  to  have  minimum  RMS  errors  in  A,  f»(s)  must  be 
adjusted  so  that  the  lags  in  following  changes  in  the  spectra 
are  balanced  by  the  fluctuations  in  the  noise  out  of  the 
detectors.  From  this  criterion  we  can  determine  the  adaptive 


loop  band-pass  as  jo. 


where  1/T 


d/d  is  the  effective 


time  constant  relating  to  the  change  in  input  spectra.  How¬ 
ever.  if  step  changes  in  the  input  signals  are  contemplated 
then  the  adaptive  loop  should  be  as  tight  as  stability  dictates 
and  the  gain  settings  in  the  figure  are  correct.  The  ratio  of 
the  filter  outputs  minus  one  is  proportional  to  6A/A  so  that 


(6  .  b)  X  ~  A  .j  A 


and  the  band-pass  in  the  adaptive  loop  is  proportional  to  that 
in  the  main  loop. 

In  all  of  the  adaptive  systems  considered  it  is  easy 
to  specify  the  gain  changes  to  keep  the  loop  dynamically  similar 
for  different  inputs,  but  it  is  harder  to  specify  the  exact  band¬ 
pass  or  the  shape  of  the  filters  in  the  adaptive  loop.  It  is 
always  possible  to  make  a  linear  stability  analysis,  if  the  in¬ 
puts  are  fixed,  and  the  noise  out  of  the  squaring  circuits  can 
be  computed  for  Markovian  noise  but  no  general  theory  of  optimal 
design  exists. 

There  are  many  ways  of  instrumenting  adaptive  servos 
which  give  adequate  performance  and  the  effects  of  non-! inear ites 
and  complexity  must  be  considered  carefully  if  a  satisfactory 
design  is  to  be  obtained.  For  example  the  use  of  smoothed 
absolute  value  instead  of  RMS  leads  to  only  a  few  per  cent  more 
noise  in  the  adaptive  loop.  As  another  example  the  division 
in  the  last  example  may  be  replaced  by  a  subtraction  if  the 
dynamic  range  is  not  too  great.  At  low  input  signals  the  loop 
is  then  sluggish  but  the  tracking  error  is  small  due  to  the 
sma 1 1  input . 

Underlying  the  design  of  adaptive  servos  is  the 
assumption  of  relatively  slow,  or  only  occasional,  changes 
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in  environment.  I  *  rapid  changes  occur  a  different  main  loop, 
possibly  non-linear,  is  required.  Adaptive  loops  may  be  called 
quasi-linear  but  because  they  are  non-linear  no  general  method 
of  analysis  has  emerged  to  determine  optimal  performance  as  a 
standard  of  comparison  with  specific  loops,  or  to  check  in¬ 
strumentation  approximations. 


t 

? 


167  - 


I 


fH|  ,OMNl  JMVIlVTf 

APPiltD  PHYSICS  l AlOIATOiY 

Hi***  «WO  HAlUANC 


Tm  to nh»  \p0vm1ry 

AMUIO  SICS  lAK>CATO«» 

um  in**) 


Fig.  4  RESULTS  OF  FIG.  3 


n«  om  <mn  \p+mvn 
AmKD  PHYSICS  UJOtATO «Y 

Mvtl  MMO 


OPTIMAL  FILTERING  IN  MISSILE  GUIDANCE 
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A.  George  Carlton 
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I  INTRODUCTION 


The  guidance  and  control  system  of  a  guided  missile 
consists  of  three  basic  elements:  a  guidance  intelligence 
system,  a  guidance  computer,  and  an  autopilot.  The  guidance 
intelligence  system  measures  in  a  suitable  reference  frame 
work  the  position  of  the  missile  relative  to  the  target,  the 
autopilot  causes  the  missile  to  execute  maneuvers  as  com¬ 
manded,  and  the  guidance  computer  commands  maneuvers  on  the 
basis  of  inputs  received  from  the  guidance  intelligence  sys¬ 
tem.  1 


The  guidance  computer  cannot  simply  command  an  ac¬ 
celeration  proportional  to  the  measured  error  in  missile 
position,  as  the  resulting  change  in  missile  position  would 
affect  the  measured  error  so  as  to  result  in  harmonic  oscil¬ 
lation  of  the  missile .  This  can  be  corrected  by  adding  to 
the  command  a  term  proportional  to  the  error  derivative,  damp¬ 
ing  the  oscillation.  With  proper  choice  of  the  constants,  such 
a  guidance  computer  may  cause  the  missile  to  respond  very 
faithfully  to  the  inputs  from  guidance  intelligence.  This  is 
net  typically  sufficient  to  result  in  the  smallest  possible 
misses,  however,  because  the  guidance  intelligence  is  not 
perfect.  In  addition  to  genuine  information  as  to  error  of 
missile  from  target,  called  the  signal,  the  guidance  intelli¬ 
gence  gives  spurious  information  called  noise.  There  are 
many  sources  of  noise,  including  internal  noise  in  the  sen 
ing  device,  atmospheric  effects,  and  scintillations  of  the 
target.  The  missile  should  respond  to  the  signal  but  ignore 
the  noise.  The  problem  of  optimal  filtering  in  missile 
guidance  is  the  problem  of  achieving  the  best  compromise  be¬ 
tween  these  conflicting  requirements. 

It  is  sometimes  thought  that  the  noise  effects  are 
of  minor  importance.  Consider  a  very  simplified  situation 
in  which  the  only  elements  are  acceleration  capability  of  the 
target  and  the  level  of  noise  which  is  "white",  i.e.,  uni¬ 
formly  spread  over  all  frequencies.  Simple  dimensional  an¬ 
alysis  shows  that  the  minimum  mean  square  miss  is  proportional 
to  the  4/5  power  of  the  noise  spectral  density,  and  a  little 
more  effort  shows  that  80  per  cent  of  the  minimum  mean 
square  miss  is  due  to  following  the  noise. 


1)  Presented  at  Indianapolis  Meeting,  AAAS,  28  December  1957. 
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The  very  similar  problem  in  directing  antiaircraft 
gunfire  was  one  application  considered  by  Wiener  in  "Extra¬ 
polation,  Interpolation,  and  Smoothing  of  Stationary  Time 
Series"  Wiener  considered  signal  and  noise  ensembles 
which  were  stationary  stochastic  processes  and  determined 
the  realizable  linear  filter  which  minimized  the  mean  square 
miss.  If  the  signal  and  noise  are  uncorrelated,  the  char¬ 
acteristics  of  interest  are  the  power  spectral  densities 
S(u)  and  N(u)  representing  the  resolution  of  the  mean  square 
signal  and  the  mean  square  noise  into  a.,  continuum  of  fre¬ 
quency  components.  Thus  E  [signal] 2  S(u)  du.  The  linear 
filter  is  defined  by  the  transfer  function  F(u)  specifying 
the  scale  factor  and  phase  shift  applied  to  each  frequency. 
The  mean  square  error  o 2  is  given  by 


(1)  o2  =  /{  |  1  -  F  |  2  S  +  |F|  2N  }  du  . 

since  J  1  —  F I  is  the  erroneous  factor  applied  to  signal  com 
ponents  and  ]f[  the  erroneous  factor  on  noise  components. 

To  be  realizable ,  the  filter  must  operate  only  on  inputs 
from  the  past,  not  on  future  inputs.  The  realizable  F  min¬ 
imizing  o 2  was  found  by  Wiener  to  be 


(2) 


_ 1 _ 

:;j;  [  S(u)  +  N(u)  ]  + 


-iut 


dt 


03 


-CD 


S(u)  eiUt  du 
(S(u)  +  N(u)j- 


In  this  expression,  S(cj)  +  N(u)  is  factored  into  (S  +  N)  + 

(S  +  N)~,  (S  +  N)+  being  analytic  and  v/ithout  poles  or  zeroes 
in  the  lower  half-plane,  with  (S  +  N)-  the  complex  conjugate 
of  (S  +  N)+ . 

These  results  have  been  applied  directly  to  missile 
guidance  problems,  but  have  some  serious  shortcomings.  The 
most  obvious  defect  is  chat  the  spectral  density  of  the  signal 
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is  assumed  iixed  and  known  Since  an  enemy  interested  in 
increasing  our  error  may  select  the  signal  characteristics, 
it  is  reasonable  to  assume  varied  techniques  chosen  to  im¬ 
pair  the  performance  of  our  system 

II  FILTERING  PROBLEM 


The  problem  of  filtering  is  essentially  one  of 
statistical  decision.  Statistical  decision  theory  in  this 
problem,  whether  we  are  fighting  a  conscious  enemy  or  im¬ 
placable  nature,  calls  for  selecting  the  filter  for  which 
the  maximized  over  possible  signal  spectra  is  a  minimum. 

In  view  of  the  complexity  of  Eq.  (2),  the  deter¬ 
mination  of  this  minima:;  filter  appears  difficult  The 
minimax  filter  can  be  determined  without  too  much  trouble, 
however,  by  the  use  of  indirect  approaches.  The  first  step 
in  solving  the  problem  is  to  investigate  the  maximin  S(<j)  , 
that  is,  the  S(u)  which  maximizes  the  minimum  achievable  o2 . 

The  possible  signal  spectra  will  be  limited  by 
linear  restrictions ,  limits  on  the  power  of  linear  functions 
of  the  signal.  An  airplane,  for  example,  will  have  some 
bounds  on  its  position,  velocity,  and  acceleration,  due  to 
limits  on  its  course,  propulsion,  and  structure.-  If  the 
mean  square  acceleration  cannot  exceed  a2,  then  I  S(u)  <j4 

2  ^ 
du<  a  .  In  general,  there  may  be  n  such  linear  restrictions 

of  the  form 


(3) 


k  .  J S(w)  Oj  (u)  du  =  1  , 


with  k  > 
J 


0,  S(u)  >  0 


Vu) 


>  0 


For  example,  with  limited  mean  square  velocity  and 
acceleration,  we  might  have  =  u2,  9^  =  u4  , 

The  maximin  S(u)  is  that  S(u)  subject  to  these  con¬ 
straints  which  maximizes 
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Introducing  the  constraints  by  Lagrange’s  method, 
we  wish  to  maximize 


(5) 


's.n  -  T  °l»  *  rxi*JSM  8j(u)  * 

-/{ Ifs,nI  2"  +  [|>  -  fs,n  |  2  '  ? 


i.k.e  sf 
j  j  jJ  J 


du  . 


For  any  increment  a  on  S .  the  resulting  increment  on  I  is 

J 


(6)  5  I 


-/fl 


Fg  +  *1  N  N  +  |l-rs  +  ^,N 


(S  +  ^)J  du 
2 


-/{ |FS.«  I  » 


1  -  F. 


,N  I  2  <S  *  *>} 


du 


+JA[ I1  -  FS,N  I  2  -  ?  Vj9j} 


du 


For  a  maximin  S(u)  it  is  required  that  6  I  be  nonpositive 
for  all  permissible 

2 

The  first  integial  is  the  minimum  o  given  spectra 
S  +  ^  and  N,  the  second  integral  and  o 2  given  the  same  spec¬ 
tra  and  nonoptiMal  F,  so  the  difference  of  the  two  integrals 
is  nonpositive.  Thus  51  is  not  greater  chan  the  third  in¬ 
tegral.  This  integral  will  be  nonpositive  if  the  coef¬ 
ficient  of  ^  is  zero  when  a  A  of  either  sign  is  permissable 
and  nonpositive  when  any  permissible  A  is  non-negative. 

Since  S(u)  >  0,  permissible  ^  are  of  either  sign  for  S(u) 

>  0  but  are  non-negative  when  S(u)  -  O. 
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(7) 


Thus  the  maximin  S(o)  is  SQ(u)  such  that 


1 


So,N 


k  e 

j  j  j 


x  k.e . 

j  j  j 


o  :  S  (u)  >  0 
o 

u  :  SQ(u)  -  0, 


with  the  Xj  chosen  to  satisfy  the  given  constraints.  The 
result  just  obtained  is  unsatisfactory  in  that  the  con¬ 
straints  were  applied  strictly,  rather  than  as  inequalities 
By  similar  manipulation  one  can  show  that  when  the  unique 
constraints  are  replaced  by  inequality  constraints,  k j  >  1 , 
the  result  is  the  same  as  above,  together  with  the  condition 


(»)  Xj  (kj  -  1)-  0, 


implying  that  each  constraint  is  either  redundant  or  ap¬ 
plied  strictly. 

To  derive  the  minimax  filter,  we  show  that  the 
game  is  determined  since 


min  max  2  <  max  2 

F  S  S,N,F  S  S,N,F„  . 

j  9 


(9) 


max 

d 


do  = 


2 

do  + 


ma;;  min  2 
S  F  S,N,F 
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2  2 
Thus  the  minimax  o  is  equal  to  or  less  than  the  minimin  o  . 

Since  the  fundamental  theorem  of  game  theory  states  that  the 

minimax  is  equal  to  or  greater  than  the  maximin,  the  game  is 

determined,  with  F„  the  minimax  filter. 

So’N 

The  second  indirect  approach  is  to  deal  with  the 


function 


1  -  F 


S  ,N 


which  is  much  more  tractable  than  F 


or  the  formally  equivalent  1  -  Fc  M 

o  rt  i  j  o  .  a j  N 

F 


S ,  N 

Applying  variational 


procedures  to  o2  - J |  F  2  N  +  |  lj"  -  f|2Sj  dui,  one  can  obtain 
the  result  derived  by  Wiener  for  the  optimal  realizable  Fg  ^ 


and  also  the  useful  result: 


(10) 


1 


2  (S  +  N) 


ReNF  +  Real  complement  to 


Im  NF 


I 


where  the  real  complement  is  the  real  function  which  must 
supplement  the  imaginary  function  to  yield  a  realizable  trans¬ 
fer  function.  For  simple  forms  of  N,  Eq  (10)  can  be  solved 
rather  easily,  for  example 


1  FS,N 


(11) 


(S  +  N) ,  -  N 


N  (a))  constant 
2 


r  -  -i  ca 

N  1  -  F(ia)  ,  N(u>)  -  g  2 

J  a  +  to 


N  +  K , 


N(u>)  "  CQ  +  C2co2  +  C4  U)4 


(K  determined  by  J"  log 


1  F 


S  ,  N 


dco  -  o) . 


Divergent  spectra  play  a  useful  role  in  some  prob¬ 
lems  and  the  general  results  on  optimal  filters  are  still 
valid. 


sible  to  determine  S  (ta)  and 

o 


By  using  the  results  Eq .  (7)  and  (11),  it  is  pos- 

1  F  12 
So'N 

filter,  and  one  can  then  determine  Fg  ^ 

special  cases  or  in  general  by  applying  Wiener's  result  to 

the  determined  S  (cn). 

o  ' 


for  the  minimax 
by  factorization  in 
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To  illustrate  the  method,  consider  the  case  of 
white  noise  of  spectral  density  N  and  mean  square  target 
acceleration  equal  to  or  less  than  a^.  Our  constraint  is 
then 


?/ 


s  (CD)  05  dco  -  1 , 


From  Eq .  (7) , 

1  -  F 


4 

X  “ 

A  ~2 
a 


cd:  S(cd)  >  0 


GO 

“2 

a 


cd  :  S(go)  -  0 


From  Eo .  (11) , 


1  -  F 


Thus,  setting  a  /X  -  oo 


N 

5~TTT  ‘ 

4 


N 


GO 


GO 


cd  :  S(u>)  >  0 


cd  :  S(go)  -  0  . 


The  solution  for  S  is 


S  -  N 


oo' 


i  CD 


“  1  , 


2  2 

CD  <  C0o 


GO  ^  G0_ 


1  r  4 

o)Q  must  satisfy  the  relation  — J  S(cd)  go  dco  -  1 ,  or 


GO, 


-  J  N  (cdo4  -  go4)  dco  -  a2,  from  which  goq  «  ^ 


2  1/5 


-  0), 
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It  is  clear  that 


1 


F 


2 


4,4  2  .  2 

0)  /0)o,  cu  <  0>o 

.  2  2 
1  .  0)  CD 

’  o 


Some  numerical  calculation  is  required  to  obtain  F  itself. 

The  minimax  mean  square  miss  is  obtained  from  the 
relation,  valid  for  white  noise, 


o2  =  -  J  N 


log 


1  -  F 


S ,  N 


dcD,  from  which  we 


2 

=  8  N  o)Q  .  The  portion  of  o  due  to  failure 


signal  is  J  S  jl  -  F 


do)  -  8  N  0)  /5, 
o 


find  o2 
to  follow  the 


From  the  fact  that  J  1  —  F  |  is  unity  for  all 
large  od,  it  follows  that  F(cd)  is  proportional  to  1/cd  for 
large  frequencies.  From  the  practical  standpoint  of  guided 
missile  design,  this  is  most  unfortunate,  as  the  mean  square 
acceleration  of  the  missile  resulting  from  noise  is 


/ 


N 


2 


4 


CD 


do) 


which  clearly  does  not  converge  for  N  constant  and  F  of  order 
1/ CD. 

It  is  therefore  necessary  to  modify  our  problem  so 

as  to  select  a  minimax  filter  among  those  for  which  J  N  |f|  ^  co* 
do)  is  limited  to  a  specified  value.  To  solve  this  problem 
we  observe  that  the  previous  determination  of  the  maximin  S  ,J 
is  valid  regardless  of  the  class  over  which  F  is  minimized, 
so  we  need  merely  find  the  optimal  F  among  those  filters  for 

which  J N  I f|  ^u^jrto)  does  not  exceed  a  specified  value.  The 
quantity  to  be  minimized  becomes 


°S,N,F  +  X 


/N 


2  4, 

CD  dCD 


/{ 


1  -  F 


+  N 


f|  2  (l+Xco4)J  dcD, 
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with  X  chosen  to  satisfy  the  mean  square  missile  acceleration 
requirement.  This  is  exactly  equivalent  to  the  former  fil¬ 
ter  optimization  problem  with  an  effective  noise  spectral 
density  of  N  (1  +  X  oq4)  .  It  is  to  solve  problems  of  this 
sort  that  one  considers  non-convergent  noise  spectra,  such 
as  N  -  Nq  +  Cg  £UZ  +  004.  Calculations  of  minimax  filter 

for  such  cases  have  been  carried  out.  The  optimal  filters 
and  the  minimax  mean  square  miss  depend  on  the  ratio  of 
possible  mean  square  target  acceleration  to  allowable  mean 
square  missile  acceleration  due  to  noise.  This  ratio  should 
be  noticeably  larger  than  unity  to  avoid  significant  de¬ 
crease  in  accuracy. 

A  completely  different  approach  to  the  problem  is_ 
hinted  at  by  some  of  the  results  obtained  for  optimal  1 1  —  F] 

(S  +  N) .  This  quantity  is  the  spectral  density  of  the 
difference  between  the  input  and  the  output  of  the  filter,  a 
01  ference  which  is  the  error  signal  in  a  servo  type  filter. 
The  spectral  density  of  this  difference  is  equal  to  the 
noise  spectral  density  if  the  noise  is  white  and  the  filter 
is  optimum.  Useful  but  less  simple  relations  can  be  de¬ 
rived  for  other  types  of  noise.  Thus  we  have  the  possibility 
of  checking  the  correctness  of  the  filtering  process  by  in¬ 
vestigating  the  spectrum  of  this  difference,  and  adjusting 
the  filter  accordingly.  An  advantage  of  this  type  of  fil¬ 
tering  is  that  it  can  take  advantage  of  a  situation  in 
which  the  opponent's  strategy  is  poor.  With  sufficiently 
slow  adaptation,  such  a  filter  will  closely  approximate  the 
minimax  filter  in  the  most  difficult  situation,  and  can  be 
considered  a  sub-minimax  filter,  one  of  the  few  practical 
examples  of  the  existence  of  sub-minimax  solutions  to  game 
theory  problems. 

Adaptive  filters  based  on  these  considerations  have 
been  devised  and  have  demonstrated  excellent  performance, 
although  there  is  at  present  no  adequate  theory  in  this  field. 
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I.  SUMMARY  OF  USEFUL  FORMULAS  IN  VECTOR  ANALYSIS 


Vector  Representation - Let  x,  y,  z  be  a  right-handed^ 

three-dimensional  rectangular  coordinate  system.  A  vector  V© 
can  be  represented  as  an  ordered  triple  of  numbers  correspond¬ 
ing  to  the  coordinates  of  the  point  p.  Two  concepts  charac¬ 
terize  a  vector:  magnitude  and  direction.  A  vector  can  be 
thought  of  as  a  directed  line  segment,  i.e.,  with  a  head  on 
one  end,  and  a  tail  on  the  other.  Translations  of  the  vector 
do  not  alter  the  vector.  Thus,  if  the  tail  is  located  at 
(x^,  y^,  z^)  ,  the  head  at  (Xg,  Yg »  z2^  •  vector  is  repre¬ 

sented  by  the  ordered  triple  (x2_xi»  yg-y^ ,  z2“zi)»  which  does 
not  depend  on  where  the  tail  is  located. 


A  more  useful  representation  of  a_^vec tor  is  based  on 
the  concept  of  base  vectors.  Let  i  ,  j  ,  k  denote  the  vectors 
whose  ordered  triple  representations  are  (1,  0,  0) , (0,  1,  0) , 
(0,  0,  1),  respectively.  Such  a  triad  is  called  a  right- 
handed  orthogonal  triad  of  unit  vectors,  and  forms  a  coordi¬ 
nate  system.  In  terms  ol  these  unit  vectors,  the  vector  V^ 
above  is  expressed  (0,  1,  0)  as 


(i  i)  V 


x  i+y  j+zk 
o  o  o 


(1,0,0) 


Length^of  a  Vector - The  length  of  the  vector  V  , 

denoted  by  ~VQ  |  ,  rs  given  by  ° 


(1.2) 


J  2  2  2 

. 

o  o  o 


-  185  - 


i 


1W4  >OI**  M<W(^ 

Am«0  PHYSICS  LAftOtATOCV 
t««w  y«Mb 


Addition  of  Vectors - If  V  -xi+yi+zk, 

- - - - - -  o  o  JoJ  o’ 

V,  -  x.i  +  y.j  +  z.k,  then  V  +  V,  is  defined  to  be  a 
1  1  1  1  ’  o  1 

vector  given  by 

(1.3)  Vq  +  vx  “  (xq  +  xx)  ~i  +  (yQ  +  yx)  t  +  (zq  +  zx)k* . 

This  definition  is  in  agreement  with  the  parallelogram 
law  found  in  most  elementary  physics  texts. 

Multiplication  of  a  Vec tor  by  a^  Scalar  (Number) - 

aVQ  is  defined  to  be  the  vector  given  by 


(1.4)  a/VQ  -  apcQi  +  <xyQj  +  c^k  . 


Exer c ises  :  Prove 


(a)  VQ  +  Vj  -  Vj  .  Vc 

(b)  tQ  ♦  (V>  ♦  V^)  -  (7  +Vj)  +  72 

(c)  or  (7q  +  fy  -  0(7  + 


Do_t^  Product  (or  Scalar  Pro  due  t )  of  Two  Vectors - The 

dot  product  of  two  vectors  is  a  scalar. 


It  is  denoted  by  V 


V^,  and  is  defined  by 


(1.5)  V  .  V  -  |  VQ|  |  V ^ |  cos  (6  ,  where  is  the  angle 

between  the  two  vectors. 


It  can  be  shown  that 


(1.6)  ~V  •  ~V^  -  x  x.  +  y  y,  +  z  z. 

'  '  o  1  ol  'o  1  ol 
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Cross  Product  (or  Vector  Product)  of  Two  Vectors - 

The  cross  product  of  two  vectors  is  a  veclor. 

It  is  denoted  by  VQ  x  V^ ,  and  is  defined  by 
(i.7)"vo  x  Vj  -  I  Vo|  |vi|  Sin  *  n^ 

where  d  is  as  above.  ^  is  a  unit  vector  normal  to  the  plane 
formed  by Y  and  V^ ,  its  direction  determined  by  the  right- 

hand  rule.  It  can  be  shown  that 


(1.8)  Vq  x  -  (yQ  zL  -  zq  yi)“i  +  (zq  xx  -  xx  xq) 


y  + 


(x  y,  -  y  x, )  k 
'  o  1  o  1' 


or,  in  the  form  of  a  determinant, 

t  t 

i 

(1.9)  VQ  x  Vx  - 


k 

z 


Components - The  component  of  a  vector  in  a  direction 

is  a  vector  formed  by  projecting  the  vector  onto  a  line  point¬ 
ing  in  that  direction.  Thus,  the  component  of  V  in  the  i 

direction  is  x  i. 

o 


Exercises : 

prove 

(d) 

v  . 
o 

V1  -  V1  •  Vo  • 

— ►  — ►  — ►  — ► 

(e) 

\  • 

<V1  +  V2>  -  Vo  ■  V1  +  Vo  •  V2  • 

— ►  “ *  ~ ►  “► 

(f) 

“<Vo 

•  vx)  -  (cyvo)  •  vx  -  vo  .  (ONJ 
-*  -*  -> 

(B) 

V  x 
o 

V1  -  -(Vx  x  Vo)- 

t 

» 

t 

I 
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(h)  a(V*o  x  Vj)  -  (aV^)  x  "v  j  =  V*Q  x  (aV^). 

“►  -►  ->  4  ->  -4  -►  — ► 

(i)  VQ  x  (Vj  x  V2)  -  (Vo  .  V2)VX  -  (V0^V1)V2. 

(!•)  VQ  x  (Vj  +  V2)  -  <V0  x  Vj)  *  (V0  XV 

<j)  v!,  ■  (V,  X  V.)  ■  (7  x?)  .?  ■ 


x  y 

o  Jo 


x2  y2  Z2 


(This  expression  is  some¬ 


times  called  the  box 


product  of  three  vectors.) 


(10  The  component  of  Vq  in  the  Vj  direction  (if 
|Vj|  /  0)  is  given  by 


(1)  V, 


i  =  ;:  ,  V  .  j  =  y  ,  V 
o  o  J  Jo  o 


can  write  V  =  (V 
o  o 


k  a  z  ,  so  that  one 
o 

-4  — ►  — ►  — ►  -4  ►  — V 

i ) i  +  (V  .  j)j  +  (Vn  .  k)k. 


(m)  Let^,  ^2  non-collinear ,  and  different  from 


zero 


Let  V  be  the  vector  formed  by  projecting  V  into  the 

O  fc  o 


plane  formed  by  Vj  and  Vg •  Then 

=  ~rT  x  (\^  x  ~n) ,  where  ~n 
J  o 


V!  *  V2 


The  following  are  in  general  impossible  or  incorrect 
statements.  Why? 


(n)  V  x  (V,  x  VJ  =  (V  x  V.)  x  V„. 


o 


(o)  VQ  .  (Vj  xf2)  -  (Vo  .  tj)  x  t2. 


(p)  Vn  .  (Vj  .  v9)  =  (V 


o 


o 


Vj)  .  V2. 
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II.  ROTATED  COORDINATE  SYSTEMS 


Let  i^  ,  j^,  denote  a  right-handed  orthogonal  triad 
of  unit  vectors  distinct  from  T,  7>  k*.  Expressing  each  , 
7r  7.  as  the  sum  of  its  components  along  7,  7»  k»  we  can 
write 


(2.1)  i 


i  +  a10  j  +  a. 


h  =  a12 


i  +  a 


22  J  +  a23 


ki  =  an  1  +  J  + 


Since  (ij  =  1,  |j1|  -  1,  ^ 

we  have 

•  *  °,  ij  x  jj  *  kx, 

(2.2) 

2  2  2  . 

“ll  +  a12  +  al3  lf 

(2.3) 

a2l  +  a22  +  a23  =  1 ’ 

(2.4) 

all  a21  +  al2  a22  +  a13  a23 

-  o. 

(2.5) 

a31  =  a12  a23  "  a22  a13’ 

(2.6) 

a32  =  “l3  a21  “  all  a23’ 

(2.7) 

a33  =  “ll  a22  ~  a21  “12  ' 

j 

Conversely,  it  can  be  shown  that  any  set  of  o's  satis¬ 
fying  Eqs .  (2.2)  -  (2.7)  will  produce  a  right-handed  ortho¬ 
gonal  triad  1^,  *  k^  when  substituted  into  Eq.  (2.1). 

Ijf  the  a’s  are _^s  a^ove,  and  one  solves  Eq.  (2.1)  for 
i,  j,  1;  in  terms  of  i^,  j^,  1?^,  the  solution,  as  can  be 

shown,  is  expressible  in  the  following  simple  form: 

(2.3)  t  -  an  Tj  +  a21  ^  +  a31  \ 

j  -  a12  tj  +  a22  ^  +  a32  kj 

— ► 

k  -  a,  o  I.  ^  flno  .l-i  ^  k. 


ii3  n 


'23  jl  +  a33  kl 
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Equations  (2.1)  and  (2.8)  make  the  problem  of  expressing 
in  a  second  coordinate  system  a  vector  known  in  one  coordinate 
system  quite  simple: 


Thus 

(2.9)  r 


o 


■  (x 


i 

0 

>» 

+ 

j 

+  z  k 
o 

=  ; 

X 

o 

(“n  h 

-* 

o 

(a12 

il 

+  a22 

ji 

+ 

“32  kl) 

-¥ 

— ► 

o 

(a13 

X1 

+  a23 

ji 

+ 

“33  kl> 

a 

11  + 

yo 

“12  + 

z 

o 

a 

13  1 1 

+  “21  J1  +  “31  kl> 


+  <xo  “21  +  “22  +  zo  “23^1 

+  <Xo  “31  +  yo  “32  +  zo  “33)kt- 


III.  DERIVATIVES  Q£  VECTORS  AND  MOVING  COORDINATE  SYSTEMS 

Suppose  the  vector  V  is  a  function  of  time.  Since  a 
vector  can  be  described  by  its  components,  it  is  clear  that 
a  situation  could  arise  where  the  components  are  constants 
in  one  coordinate  system,  yet  varying  in  another  coordinate 
system  moving  with  respect  to  the  first.  To  talk  about  the 
derivative  of  a  vector,  we  must  then  first  specify  the  co¬ 
ordinate  system.  In  Newtonian  Mechanics,  most  physical  laws 
expressible  in  vectorial  form  achieve  their  simplest  ex¬ 
pressions  when  the  coordinate  system  is  fixed  relative  to 
the  fixed  stars  (or  moving  with  a  uniform  velocity  with 
respect  to  them).  (In  a  gocd  many  missile  systems,  the  co¬ 
ordinate  system  may  be  fixed  relative  to  the  earth,  as  the 
motion  of  the  earth  has  negligible  effects).  We  therefore 
define  the  derivative  of  a  vector  V  (more  properly  the  in¬ 
ertial  derivative )  Os  follows: 

=  xi  +  yj  +  zk,  where  i,  j,  k,  have  fixed  direc¬ 
tions  in  inertial  space,  then 

(3.1)  =  xi*  +  yj*  +  ziT. 

An  alternative  definition,  equivalent  to  the  above,  is 

(3.2)  v'(t)  =  lim  . 

At-K) 
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V,  being  a  vector,  can  be  expressed  in  a  second  coor¬ 
dinate  system  bv  the  rules  of  Part  II.  Care  must  be  taken 
that  the  1,  j ,  k  in  the  abovedef inition  are  inertlally 
fixed,  for  if  V  =  i^  +  k^,  and  TL,  3^*  are 

in  motion,  it  is  not  true  that  =  x^  +  y^  +  Z1  "^l  • 

(see  exercises  below) 


Exercises:  Prove 


(q) 

d 

dt 

(at)  ■ 

•  at 

+  at 

(r) 

d 

dt 

(V*  ♦ 

4 

'  V1 

(s) 

d 

dt 

^  • 

■  K 

(t) 

d 

dt 

x 

■  *i ; 

(u) 

If 

T, 

,  It. 

form 

of  unit  vectors,  and  tT  =  x^  7^  +  ^1  ^1  +  Z1  kl  ’  then 


r=  x. 


+  yx  +  z 


+  x,  + 


i  “i  '  "i  *i  '  yi  J i 


+  zi  ki 


Let  i, ,  jT ,  k  be  a  right-handei  orthogonal  triad  of 
XIX 

unit  vectors,  in  motion  with  respect  to  the  inertially  fixed 
triad  i,  J,  k.  If  one  differentiates  Eq.  (2.2),  one  obtains 

(3.S)  «n  iu  +  al2  i12  +  «13  i13  -  0 

which  is  equivalent  to 


(3.4)  i. 


1^=0. 


Since  then  i.  is  normal  to  i. ,  it  can  be  expressed  as 

a  linear  combination  of  j.  and  k.  .  There  then  exist  functions 
a  and  b  such  that 

(3.5)  i^  =  ajj^  +  bk^.  Similarly,  there  exist  functions  c,  d, 
e,  f,  such  that 


(3.6)  ~J1 


=  ci^  +  dk^ 
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(3.7)  k. 


ei,  +  fj 


Differentiating  Eq .  (2.4),  the  following  string  of 
equalities  can  be  seen  to  hold: 


(3.8)  a  =  ?.  .  j* 


all  a21  +  a12  a22  +  a13  a23 


(flEll  a21  +  a12  a22  +  a13  a23) 
=  -  (?x  •  jj)  =  -  c. 


Similarly,  it  can  be  shown  that  b  =  -  e  and  d  =  -  f. 

Setting  a  *  a>3 ,  e  =  a>2>  d  “  ^ »  Eqs.  (3.5)  -  (3.7)  can 
be  rewritten  as 

(3.9)  fj  -.3?,-  <*>2  kj 
(3  10)  ^  =  -  ^3  +  i)!  1?! 

(3.11)  ifj  ■  X2  • 

If  we  introduce  the  vector 

(3.12)  t  =  a»!  ?!  +  ?  +  ^3 

note  that  where  0  is  the  angular  velocity  vector,  Eqs.  (3.9)  - 
(3.11)  are  equivalent  to 

(3.13)  ?!  =  ?  x  ? 

(3.14)  ^  =  Q  x  j ! . 

(3 . 15)  1^!  =  sT  x  1<!  . 

The  vector  IT  is  called  the  angular  velocity  vector  of 
the  moving  coordinate  system.  Its  physical  interpretation  is 
that  its  direction  gives  the  direction  about  which  the  coor¬ 
dinate  system  is  instantaneously  rotating,  its  magnitude  gives 
the  rate  of  rotation,  the  sign  of  rotation  determined  by  the 
right-hand  rule. 
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The  relationship  between  the  (incorrect)  derivative 
as  computed  by  an  individual  fixed  in  a  moving  coordinate 
system  and  the  inertial  derivative  can  now  be  expressed  as 
^  • 

(3.16)  V .  .  .  ,  -  ^  .  .  +  if  x  \f. 

inertial  moving  observer 


for 

(3.17)  V 


inertial 


xi  ti  +  yx  J*i  +  +  xx  (*  x  T1) 

+  y.^  (^  x  j^)  +  Zj,  (^  x  1^) 


-5r 


moving  observer 


^  -►  — ►  -y 

+  f)  x  (xx  ix  +  y1  +  z1  kx) 


=  V  +  x  V 

moving  observer 


Exercise 


(v)  Prove 


inertial 


•p  ^  ■> 

V  +  2(]  x  V 

moving  observer  moving  observer 


+  (T  X  ^  X  (ft  X  \T)  . 


where 

rf  _  •  •  -7*  “V 

Vmoving  observer  X1  11  +  yl  ^ 1  +  Z1  K1 ’ 


IV.  MISSILE  MOTION 

Let  us  consider  the  problem  of  computing  the  position 
of  a  missile  of  the  cruciform  type  in  space  as  a  function  of 
time.  Such  a  problem  would  be  of  importance  in  a  trajectory 
simulation. 

Let  T,  jf,  £  be  an  inertial  orthogonal  triad,  pointing 
up.  (We  assume  that  the  earth  is  flat,  and  we  neglect  earth 
motion)  . 


193 


1 1 


tn*  jo» m  norm  utrmmrt 
APPUtD  PHYSICS  LAtOtATOtY 

MW  MNO  MAtTUM 


Let  denote  the  vector  joining  some  fixed  point  (say 
the  launch  site)  to  the  center  of  gravity  of  the  missile,  and 
let 

(4.1)  ?  -  xT  +  yj*  +  ziT. 

$  is  called  the  missile  position  vector.  The  motiva¬ 
tion  of  this  section  is  to  find  P  as  a  function  of  time. 


The  missile  velocity  vector  with  respect  to  the  launch 
site  is  given  by 

(4.2)  ^  -  xT  +  yj*  +  zk^  and  the  missile  acceleration  vector 

by 


(4.3) 


•  • 

-T  la-k 

P  -  xf  + 


yj  +  zk. 


If  F  is  the  force  on  the  missile,  m  its  mass,  Newton's 
second  law  of  motion  tells 

(4.4)  F  -  mP. 

Assuming  we  know  we  see  that  it  is  a  simple  matter 
to  find  x'  y,  z,  and  integrating  twice  gives  us  our  answer. 
However,  F*  comes  to  us  in  missile-fixed  coordinates.  (For 
example,  thrust  is  along  the  longitudinal  axis,  lift  normal 
to  it).  So  a  coordinate  conversion  will  be  called  for. 


Let  i^,  j^,  k^  be  determined  by  missile  orientation, 


i1  parallel  to  the  longitudinal  axis  of  the  missile,  j. 


par¬ 


allel  to  one  control  surface  pair,  k^  parallel  to  the  other. 
In  general,  we  will  know  F. ,  F0,  F„  (more  about  this 

«♦  -y  1  •‘Jy  J  -y  -y 

later)  where  Eq.(4.5)  F  -  Fj^  +  Fg  j^  +  Fg  -  gm j  . 
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will  be  thrust  minus  drag,  F 2  and  F3  aerodynamic 
lifts.  The  last  term  is  just  gravity. 

From  Eq.  (2.1)  we  have 
(4.6)  *  '  "  IF^n  +  F2a21  +  F3<r31) 

y  ’  5  1  Fl“l2  +  f2“22  +  F3a321  ‘  g 

z  "  5  1  Fl“l3  +  F2a23  +  F3®33] 

Integrating  twice  gives  x,  y,  z.  It  is  clear  that  a's 
must  be  known.  That  is,  we  must  know  the  orientation  of  the 
missile  in  space. 

If  we  knew  the  angular  velocity  vector  ^  of  the  i^,  j^, 
kj^  coordinate  system,  the  a's  could  be  easily  found. 


For  example , 

-P 

(4.7)  an  -  ±l  .  i  - 


(ft  x  i^)  .  i  =  Cl  .  (i^  x  i)  * 

-t  -f 

il  J1  kl 

10  0 


all  a21  a31 


(“l1!  +  “2^1  +  Vl*  *  a31Jl  +  a21kl^ 


”  “2a31  +  ^^l 
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Proceeding  similarly,  we  have  the  equations 


all 

11 

toB 

“31 

+ 

“3 

“21 

ft- 

»-■ 

to 

“  X2 

“32 

+ 

“3 

“22 

“13 

"  “2 

“33 

+ 

“3 

“23 

a21 

=  a)  a 

1  a31 

0) 

3 

a 

11 

a22 

=  0)  a 

1  a32 

0) 

3 

a 

12 

a23  “l  “33  "  ^3  “l3 


a31  “  “  “l  a21  +  “2  “ll 


a32  "  “l  a22  +  ^2  “l2 


a33  "  X1  a23  +  ^2  “l3 


Integrating  these  equations  then  would  yield  the  a's. 
The  numbers  u^,  can  be  found  as  follows  if  we  know 

the  applied  moments  M.  ,  M„,  Mo  about  the  center  of  gravity 

which  would  cause  rotations  about  i^,  jp  respectively: 

The  angular  momentum  vector  is  given  by 
(4.9)  H  ■=  Ij  ij  +  I2  a)2  ^  +  l2  u>3  ^ 


where  I  , 
about  i^  , 


I„,  In  are  the  moments  of  inertia  of  the  missile 

Hr 

j ^ ,  kj^  respectively,  taking  the  origin  at  the  center 


of  gravity.  (For  a  general  rigid  body,  the  expression  for  H 
is  more  complicated,  involving  the  products  of  inertia.  In 
this  case,  however,  because  of  symmetry,  the  products  of  iner¬ 
tia  all  vanish,  and  the  moments  of  inertia  about  and  are 
equal . ) 
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If  we  set 

■  ^  ^  ^ 

(4.10)  M  =  ij  +  M2  j1  +  Mg  k^,  the  moment  (or  torque)  vector, 
Mechanics  tells  us  that 

(4.11)  ~a  =  M. 


Differentiating  Eq.(4.9),  we  have 

(4.12)  if  - 1.  \  +  i2a>2  jj  +  i2S  ki  +“  x  “ 

■  ll  “l  T1  +  tl2  *2  +  (I1  -  V  j*i 

+  [12%  +  “1*2  (I2  "  Il)1  ^1 

=  Mx  'il  +  M2  “j1  +  M3  kx  =  M. 

Hence 

M1 

(4.13)  0)  =  (why?) 

1  A1 


X2 


M2  “  (I1  “  I2)  “l  % 


= 

3 


M3  *  <r2  *  V  “l  “2 


Equation  (4.13)  is  a  special  case  of  "Euler's  equations". 

Integrating  Eq.  (4.13)  produces  the  CD's,  which  produce 
the  a's,  which  in  turn  produce  x,  y,  z.  Our  problem  is  now 
completely  solved  provided  we  know  F^^ ,  Fg ,  Fg,  M^,  Mg,  Mg,  m, 

I,,  I0.  The  last  three  are  trivial;  The  first  six  are  usually 

obtained  from  wind  tunnel  tests.  They  are  generally  quite  com¬ 
plicated  functions  of  other  variables.  It  is  not  our  purpose 
to  exhibit  these  functions  explicitly,  but  it  will  be  worth¬ 
while  to  at  least  have  a  look  at  what  these  other  variables 
are.  F  of  course  depends  on  thrust.  None  of  the  others  do, 
unless  there  is  thrust  misalignment.  All  six  depend  on 

(a)  the  magnitude  and  direction  of  the  air  stream 
velocity  with  respect  to  the  missile, 

(b)  speed  of  sound  (which  changes  with  temperature), 

(c )  air  pressure  (depends  on  altitude  and  temperature), 
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(d)  center  of  gravity  position  (which  moves  as  fuel 
is  spent)  and 

(e)  the  angles  <5^,  6g,  <5g,  <5^  at  which  the  control 
surfaces  are  inclined. 


In  addition  M^,  Mg,  Mg  depend  on  ,  aOg,  a>g  (aero¬ 
dynamic  damping) . 

■“V* 

A  few  words  about  (a)  and  (e)  are  in  order.  If_JW  i^ 
the  velocity  vector  of  the  wind  with  respect  to  the  i,  j, 

1?  coordinate  system,  the  P  -  if  is  the  (negative  of  the) 
velocity  of.  the  airstrean^  wit^h  respect  to  the  missile. 
Expressing  P  -  W  in  the  1*1  ,  j.  ,  1c  coordinate  system,  we 
have  ill 

(4.14)  P  -  W*  =  U  i.  +  V  j,  +  W  k.  ,  where  U,  V,  W  are  some 
numbers . 

It  is  possible  to  find  numbers  a,  &  such  that  (why?) 

(4.15)  "p  -  W  =  Ip  -  W  |  (cos  a  ^  +  sin  a  sin  0  j^ 

+  sin  a  cos  &  k^) . 

|P  -  W|  is  called  air  speed,  a  is  called  angle  of  attack, 
and  is  called  the  aerodynamic  roll  angle.  Air  speed  divided 
by  speed  of  sound  is  called  mach  number.  Data  from  wind  tun¬ 
nel  tests  often  comes  in  terms  of  mach  number,  angle  of  attack, 
and  aerodynamic  roll  angle  (other  parameters  different  from 
these  must  also  be  given,  such  as  pressure,  for  example). 


Exercise : 

-*1 

(w)  If  W=W^  i  +Wgj  +Wgk,  express  air  speed,  angle 
of  attack,  aerodynamic  roll  angle  as  functions  of  W^,  Wg,  Wg, 
x,  y,  z,  a^,  a12,  a^,  a21,  otgg,  flfgg ,  <*31»  a32>  “33- 


The  object  of  a  missile  flight  is  generally  to  intercept 
a  target.  Artistry  in  many  fields  is  required  to  achieve 
this  objective  in  the  "best"  (a  highly  subjective  term,  so  we 
shall  use  it  freely  without  attempting  to  define  it)  possible 
manner . 

The  aerodynamic  designer  tries  to  make  the  functional 
dependencies  of  F^,  Fg,  F3 ,  M^ ,  Mg,  Mg  on  the  aforementioned 


t' 


k 
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parameters  the  best  possible. 

Given  the  missile  design,  we  have  seen  how  the  missile 
motion  is  determined  once  the  missile's  environment  is  known, 
and  once  <5^,  <52>  6^,  <5^  are  known.  <5^,  6^,  6^,  6  represent 

our  control  over  the  missile.  (Thrust  can  be  controlled  in 
some  cases,  also).  Thus,  given  the  aerodynamic  design  and 
the  thrust,  the  problem  reduces  to:  How  best  should  6^,  6^, 

63,  64  be  varied  in  the  presence  of  a  varying  environment, 

and  how  does  one  do  it?  This  question  opens  a  Pandora's  box 
of  problems  which  today  are  keeping  thousands  of  people  busy. 
It  is  not  our  purpose  to  continue  much  further  along  these 
lines,  although  we  will  subsequently  discuss  certain  aspects 
which  will  have  some  bearing  on  the  matter. 


V.  TWO  DIMENSIONS 

We  have  seen  how  the  equations  for  a  portion  of  a  three- 
dimensional  simulation  can  be  set  up.  The  loop  is  nowhere  near 
complete,  however.  The  sensing  instruments,  autopilot,  guid¬ 
ance  computer,  guidance  intelligence,  as  well  as  other  features 
must  also  be  simulated.  These  features  are  discussed  elsewhere 
in  the  training  program. 

Such  a  simulation  (the  1103A  digital  computer  at  APL  is 
currently  engaged  in  such  tasks)  is  most  often  used  for  per¬ 
formance  analyses.  An  autopilot  or  guidance  computer  designer 
will,  however,  do  most  of  his  work  in  two  dimensions,  as  the 
three  dimensional  equations  are  too  complicated  to  gain  in¬ 
sight.  Occasionally  he  will  work  in  three  dimensions,  when 
his  problems  are  basically  three  dimensional  in  nature.  Most 
of  the  time  this  is  not  necessary. 


Accordingly,  let  us  reduce  the  equations  that  have  been 
derived  so  far  to  two  dimensions,  where  they  are  of  a  simpler, 
more  suggestive,  and  perhaps  a  more  familiar,  form. 


We  select  the  i,  j  plane,  and  assume  everything  happens 
in  that  plane.  For  simplicity,  assume  W  =  0.  Then 


(5.1)  z  =  0,  F3  =  0, 


0). 


axl  =  cos  y,  a12  =  sin  f »  «13  “  0 

cos  ,  a„0  m  0 


a21  =  -  sin  a22 


‘23 
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a31 

-  o, 

“32  "  °- 

a33  1  * 

M1 

■  «2 

-  0,  0  - 

7T 

-  Tj’  <5^  =  6g  =0,  <52  =  ±  64  =  6  (where 

+  or  -  depends  on  sign  convention) 


From  the  above  and  Eq.(4.7), 

(5.2)  x>_  =  it> 

Equation  (4.6)  becomec 

(5.3)  x  =  i  [  F  cos  if  -  F„  sin  if) 

ml  & 

y  =  ~  I  Fx  sin  +  F2  cos  ifr]  -  g 

Equations  (4.8)  all  reduce  either  to  0  =  0,  or  Eq.  (5.2). 

Equations  (4.13)  reduce  to 
•  M3 

(5.4)  if  =  . 

A2 

Given  the  two  forces  and  the  moment,  Eqs .  (5.3)  and 

(5.4)  enable  us  to  solve  for  x  and  y.  (Also,  of  course,  for 
if). 


Sometimes  it  is  desirable  to  write  an  equivalent  set  of 
equations  in  terms  of  y,  where  if  v  is  missile  speed, 


(5.5)  v  =~]Jx2  +  y2 

(5.6)  x  =  v  cos  y 
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(5.7)  y  =  V  sin  y. 

From  (4.15), 

(5.8)  P  -  W  =  v  cos  y  i  +  v  sin  y  j  =  v  cos  a  i^  -  v  sin  a 

=  v  cos  a  (cos  if  i  +  sin  if  j) 

-w  -V 

-  v  sin  a  (-  sin  if  i  +  cos  if  j ) 

— ¥ 

=  v  cos  (if  -  a)  i  +  v  sin  (if  -  a)  j  . 

Hence 

(5.9)  i/  =  a  +  y. 

(5.3)  becomes 

(5.10)  v  cos  y  -  vy  sin  y  =  —  [  F.  cos  if  -  F„  sin  VO 

Dl  1  Z 

v  sin  y  +  vy  cos  y  =  i  [  F^  sin  if  +  Fg  cos  VO  -  g  . 

Multiplying  the  first  equation  by  cos  y,  the  second  by 
sin  y,  adding,  one  has 

(5.11)  v  =  ^  [F1  ]  -  g  sin  y 


where 

(5.12)  F^  =  F^  cos  a  -  Fg  sin  a. 

Multiplying  the  first  by  -  sin  y,  the  second  by  cos  y, 
and  adding,  one  has 

(5.13)  vy  =  ^  [F2  ]  -  g  cos  y, 
where 

(5.14)  Fg  =  F^  sin  a  +  cos  a. 

It  is  common  for  an  autopilot  designer  to  make  further 
simplifications.  He  will  commonly  neglect  Eq .  (5.11)  entirely, 
assuming  that  v  =  0.  He  w|ll  assume  F.  and  a  are  small,  and 
replace  Eq.  (5.14)  with  F„  -  F„  cos  <z  7  He  will  neglect 
gravity,  make  first  order^assurapt ions  on  Fg  cos  a,  divide 
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through  by  v,  and  come  up  with  a  replacement  for  Eq .  (5.13), 
namely 

(5.15)  y  =  Aa  +  B6 

where  A  and  B  are  functions  of  mach  number,  speed  of  sound,  air 
pressure,  center  of  gravity,  mass. 

He  will  linearize  Eq.  (5.4),  coming  up  with 

(5.16)  ifr  =  -D^  -  Ca  +  E<5,  where  C,  D,  E  are  functions  of  the 
same  things  A  and  B  are. 

Equations  (5.9),  (5.15),  and  (5.16)  are  then  used  to  com¬ 
pute  a,  y  as  functions  of  6 .  If,  in  addition,  he  wishes  to 
know  x  and  y,  he  uses  Eqs .  (5.6)  and  (5.7). 

Exercise 

(x)  Rewrite  sectUm  V  under  the  additional  assumption 
that  W  lies  in  the  T,  j  plane,  but  is  different  from  zero. 


VI .  MEASUREMENTS  MADE  BY  MECHANICAL  END  INSTRUMENTS 

In  the  Bumblebee  family,  there  are  three  basic  de\ ices 
commonly  used  whose  measurements  can  be  simply  expressed  in 
vectorial  form  (.when  the  instruments  are  perfect,  which  of 
course  they  never  are).  These  are  the  free  gyro,  the  rate  gyro, 
and  the  accelerometer.  There  are  other  devices  used  in  these 
and  other  missiles,  such  as  stable  platforms  and  stapfus.  It 
is  beyond  the  scope  of  this  paper  to  delve  very  far  into  this 
field.  Instead,  we  shall  merely  state  what  the  above  three 
devices  measure,  and  discuss  some  of  their  applications.  We 
shall  not  describe  the  mechanical  details  of  how  they  do  this, 
leaving  this  subject  for  the  reader  to  pursue. 

The  accelerometer  as  used  in  the  Bumblebee  missiles  is 
mounted  in  the  missile,  and  is  sensitive  to  accelerations  al<jng 
a  chosen  direction  fixed  in  the  missile.  Specifically,  if  n 
is  a  unit  vector  in  this  direction,  the  accelerometer  measures 

(P  +  g  j)  .  n,  where  as  before,  j  points  up,  and  g  is  accelera¬ 
tion  due  to  gravity.  Usually,  there  are  two  accelerometers, 
one  for  which  "n  =  j ^  ,  the  other  for  which  7?  =  k..  They  are 
used  in  the  missile  autopilot .  The  acceleration  command  is 
compared  to  the  accelerometer  output,  and  the  wings  are  driven 
until  the  difference  is  zero.  An  autopilot  which  functions  in 
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this  manner  is  called  an  accelerometer  (or  acceleration) 
feedback  autopilot. 


The  rate  gyro  is  sometimes,  but  not  invariably,  mounted 
in  the  missile,  and  is  sensitive  to  rotations  about  a  chosen 
direction  fixed  in  the  missile.  Specifically,  if  n  is  a  unit 
vector  in  this  direction,  the  rate  gyro  measures  if  .  rf. 

Usually,  there  are  three  rate  gyros,  for  n  =  i^,  j ^ , 

respectively.  The  latter  two  are  used  for  damping  in  the  auto¬ 
pilot,  and  the  former  in  the  roll  control  loop,  One  will  also 
find  rate  gyros  mounted  on  homing  seekers,  whose  purpose  is  to 
aid  in  space  stabilization  of  these  devices,  as  well  as  to 
furnish  steering  error  signals  to  the  guidance  computer. 

The  free  gyro  has  two  degrees  of  freedom,  and  two  possible 
outputs.  The  free  gyro  is  used  for  attitude  stabilization  dur¬ 
ing  boost,  where  both  outputs  are  used,  and  in  the  roll  control 
loop  during  beam  riding,  where  only  one  output  is  used.  There 
are  three  definitive  directions,  denoted  by  the  unit  vectors 
S,  Gn,  G  ,  where  S  is  the  direction  of  the  spin  axis,  fixed  in 

inertial  space  by  virtue  of  the  action  of  the  gyro.  G _,is  the  t 
ipner  gimbal  axis,  and  is  always  perpendicular  to  bothAS_^and  Gq. 
Gq  is  the  outer  gimbal  axis,  fixed  in  the  missile.  Let  n  be 
some  unit  vector  fixed  in  the  missile  which  is  perpendicular,  to 
(fQ.  The  two  possible  outautss  can  be  expressed  in  terms  of  S, 

_ .  —a  SxGq 

G~,  n  since  clearly  G-  “  i  j  ■  . 

°  1  ls-G0| 

— ¥  — ♦ 

(6.1)  cos  9^  =  S  .  Gq. 

The  second  output  is  the  angle*  9_  between  GT  and  n. 

SxGq 

(6.2)  cos  9n  =  n.  —  . 

2 

In  the  roll^  control  loop,  is  usually  horizontal,  G^ 
coincident  with  i^.  Only  the  angle  Qg  is  used. 

— ►  — * 

_,In  the  attitude  stabilization  loop  for  boost,  GQ  is  j,, 
iT  is  kj^ ,  S' is  in  the  direction  of  firing. 

Exerc ise 

(y)  Prove  that  sin  9g  =  ±  '  ^ —  ,  the  sign  depending  on 

convention.  |S  x  Gq| 

♦or  some  function  of  it. 
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VII.  OTHER  USEFUL  COORDINATE  SYSTEMS 

In  this  paper,  most  of  the  useful  techniques  for  manipu¬ 
lating  coordinate  systems  have  been  presented.  Practically 
all  three  dimensional  work  with  missile  motion  can  be  handled 
by  a  judicious  use  of  the  concepts  and  basic  formulae  developed 
here.  (Theie  is,  of  course,  nothing  here  that  is  new,  and 
everything  can  be  found  in  one  or  more  well  known  texts  on 
Mechanics  or  Vector  Analysis) 

Having  developed  the  basic  concepts  and  formulae,  a  few 
applications  and  practical  manipulations  have  been  presented. 

We  c>"uld  go  on  at  considerable  length  discussing  other  appli¬ 
cations  and  special  coordinate  systems  in  detail.  This  will 
not  be  done  here,  however,  as  any  reader  will  now  have  little 
difficulty  in  mastering  such  topics,  should  he  come  in  direct 
contact  with  them. 

We  will,  however,  mention  in  passing  some  other  useful 
coordinate  systems. 

Wind-Fixed  Coordinate  Systems - It  sometimes  simplifies 

calculations  when  a  constant  velocity  wind  is  being  studied 
to  fasten  the  tail  of  the  vector  P  to  a  point  moving  with  the 
wind.  This  is  still  a  Newtonian  frame  of  reference,  so  that  Eqs . 
(4.4)  and  (4.11)  hold.  In  this  coordinate  system,  Section  V 
carries  over  intact.  (Compare  with  the  results  of  Exercise 
(x).) 


Radar-Fixed  Coordinate  Systems - Here  the  triad  of  unit 

vectors  is  fixed  in  the  dish  of  a  tracking  radar  or  a  guided 
missile  guidance  transmitter.  This  is  useful  when  studying 
tracking  radar  dynamics  or  certain  beam  riding  problems,  in 
particular  for  developing  expressions  for  beam  riding  error 
signals . 

Deck-Fixed  Coordinate  Systems - When  missiles  are  being 

fired  from  a  ship"  the  motion  of  the  ship  presents  new  prob¬ 
lems  from  the  land-based  case.  Since  radars  are  normally 
attached  to  the  deck,  their  measurements  are  most  easily  ob¬ 
tained  in  a  deck-fixed  coordinate  system.  Stabilization  of 
these  radars  is  then  required.  This  calls  for  coordinate 
conversions  from  deck  to  radar,  or  deck  to  inertial,  of  one 
type  or  another.  These  conversions  are  accomplished  physi¬ 
cally  by  the  judicious  use  of  gyroscopic  devices,  such  as 
rate  gyros  or  gyrocompasses.  (The  latter  is  a  form  of  stable 
platform) . 
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Homing  Seeker-Fixed  Coordinate  Systems - A  homing 

seeker  is  a  tracking  radar  mounted  in  the  missile,  and  prop¬ 
erly  choosing  certain  unit  vectors  fixed  in  the  seeker  aids 
in  studying  seeker  dynamics. 

Inertial  Guidance  Coordinate  Systems - In  inertially 

guided  aircraft  or  (long  range)  missiles,  it  is  sometimes 
convenient  to  choose  a  moving  coordinate  system,  one  of 
whose  unit  vectors  points  in  the  direction  of  the  normal  to 
the  earth's  surface  at  the  position  of  the  aircraft  or  mis¬ 
sile.  Sometimes  people  choose  instead  the  direction  of 
gravity,  and  sometimes  the  direction  to  the  center  of  the 
earth.  (These  three  directions  are  slightly  different). 
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